Jul 27 15:05:28.436849 ip-10-0-143-46 systemd[1]: Starting Kubernetes Kubelet...
Jul 27 15:05:28.894878 ip-10-0-143-46 kubenswrapper[2548]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 15:05:28.894878 ip-10-0-143-46 kubenswrapper[2548]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 27 15:05:28.894878 ip-10-0-143-46 kubenswrapper[2548]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 15:05:28.894878 ip-10-0-143-46 kubenswrapper[2548]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 27 15:05:28.894878 ip-10-0-143-46 kubenswrapper[2548]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 15:05:28.895487 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.894879    2548 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 27 15:05:28.902191 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902163    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 15:05:28.902191 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902182    2548 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 15:05:28.902191 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902186    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 15:05:28.902191 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902189    2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 15:05:28.902191 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902192    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 15:05:28.902191 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902195    2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 15:05:28.902191 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902198    2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 15:05:28.902191 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902202    2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902205    2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902207    2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902210    2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902213    2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902217    2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902222    2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902225    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902228    2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902231    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902234    2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902237    2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902240    2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902243    2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902246    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902249    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902252    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902254    2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902257    2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 15:05:28.902461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902260    2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902262    2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902265    2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902267    2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902270    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902272    2548 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902275    2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902277    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902280    2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902282    2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902285    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902287    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902290    2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902293    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902295    2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902298    2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902301    2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902304    2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902306    2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 15:05:28.902938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902308    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902311    2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902313    2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902317    2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902319    2548 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902329    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902331    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902334    2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902336    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902339    2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902341    2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902344    2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902346    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902349    2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902351    2548 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902354    2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902356    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902359    2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902361    2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902363    2548 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 15:05:28.903452 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902366    2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902368    2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902371    2548 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902374    2548 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902376    2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902378    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902381    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902384    2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902387    2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902389    2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902393    2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902398    2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902400    2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902403    2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902405    2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902408    2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902411    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902413    2548 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902421    2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 15:05:28.903949 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902424    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902427    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902429    2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902432    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902435    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902437    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902440    2548 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902442    2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902444    2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902447    2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902449    2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902452    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902454    2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902456    2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.902459    2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903224    2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903234    2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903238    2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903241    2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903244    2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 15:05:28.904420 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903247    2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903250    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903253    2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903256    2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903259    2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903262    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903265    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903268    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903270    2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903273    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903276    2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903278    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903282    2548 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903285    2548 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903287    2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903291    2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903294    2548 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903297    2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903300    2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903302    2548 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 15:05:28.904925 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903305    2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903307    2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903309    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903312    2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903315    2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903318    2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903320    2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903322    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903325    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903327    2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903329    2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903333    2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903335    2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903339    2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903342    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903344    2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903346    2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903349    2548 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903351    2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903354    2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 15:05:28.905429 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903356    2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903360    2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903363    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903365    2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903368    2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903371    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903374    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903376    2548 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903381    2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903384    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903386    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903388    2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903391    2548 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903393    2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903396    2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903399    2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903401    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903404    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903406    2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903408    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 15:05:28.905938 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903411    2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903413    2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903416    2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903418    2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903421    2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903424    2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903426    2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903429    2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903431    2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903434    2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903436    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903439    2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903441    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903445    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903448    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903450    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903452    2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903455    2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903458    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903461    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 15:05:28.906443 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903465    2548 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903468    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903470    2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903472    2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903475    2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903477    2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903480    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903483    2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903486    2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903488    2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903491    2548 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903493    2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903496    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903498    2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903914    2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903921    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903925    2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903928    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903931    2548 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 15:05:28.906953 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903933    2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903936    2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903938    2548 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903941    2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903944    2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903946    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903949    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903952    2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903954    2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903957    2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903959    2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903965    2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903968    2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903972    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903975    2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903977    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903980    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903982    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903985    2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 15:05:28.907422 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903987    2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903990    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903992    2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903994    2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903997    2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.903999    2548 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904002    2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904004    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904006    2548 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904009    2548 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904011    2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904014    2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904017    2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904019    2548 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904022    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904024    2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904027    2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904032    2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904035    2548 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904038    2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 15:05:28.907898 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904042    2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904045    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904048    2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904051    2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904054    2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904057    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904061    2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904064    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904067    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904069    2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904072    2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904074    2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904076    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904080    2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904083    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904086    2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904089    2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904091    2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904094    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904096    2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 15:05:28.908414 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904099    2548 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904101    2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904104    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904106    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904109    2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904111    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904114    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904116    2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904118    2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904121    2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904123    2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904126    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904128    2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904130    2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904132    2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904135    2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904137    2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904140    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904145    2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904148    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 15:05:28.908926 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904150    2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904153    2548 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904155    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904158    2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904160    2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904163    2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904167    2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904169    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904172    2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904174    2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904177    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904179    2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904181    2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904184    2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904186    2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904257    2548 flags.go:64] FLAG: --address="0.0.0.0"
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904264    2548 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904276    2548 flags.go:64] FLAG: --anonymous-auth="true"
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904280    2548 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904284    2548 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904288    2548 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 27 15:05:28.909428 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904292    2548 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904296    2548 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904299    2548 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904302    2548 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904305    2548 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904308    2548 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904311    2548 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904314    2548 flags.go:64] FLAG: --cgroup-root=""
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904316    2548 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904320    2548 flags.go:64] FLAG: --client-ca-file=""
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904324    2548 flags.go:64] FLAG: --cloud-provider="external"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904328    2548 flags.go:64] FLAG: --cluster-dns="[]"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904332    2548 flags.go:64] FLAG: --cluster-domain=""
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904334    2548 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904337    2548 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904340    2548 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904344    2548 flags.go:64] FLAG: --container-log-max-files="5"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904348    2548 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904351    2548 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904354    2548 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904357    2548 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904360    2548 flags.go:64] FLAG: --contention-profiling="false"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904363    2548 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904367    2548 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 27 15:05:28.909950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904370    2548 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904372    2548 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904376    2548 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904379    2548 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904382    2548 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904385    2548 flags.go:64] FLAG: --enable-load-reader="false"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904388    2548 flags.go:64] FLAG: --enable-server="true"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904391    2548 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904395    2548 flags.go:64] FLAG: --event-burst="100"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904398    2548 flags.go:64] FLAG: --event-qps="50"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904400    2548 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904403    2548 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904405    2548 flags.go:64] FLAG: --eviction-hard=""
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904409    2548 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904412    2548 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904415    2548 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904418    2548 flags.go:64] FLAG: --eviction-soft=""
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904422    2548 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904425    2548 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904428    2548 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904431    2548 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904434    2548 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904437    2548 flags.go:64] FLAG: --fail-swap-on="true"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904440    2548 flags.go:64] FLAG: --feature-gates=""
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904443    2548 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 27 15:05:28.910545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904448    2548 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904450    2548 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904454    2548 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904457    2548 flags.go:64] FLAG: --healthz-port="10248"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904459    2548 flags.go:64] FLAG: --help="false"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904462    2548 flags.go:64] FLAG: --hostname-override="ip-10-0-143-46.ec2.internal"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904465    2548 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904468    2548 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904471    2548 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904474    2548 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904477    2548 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904480    2548 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904483    2548 flags.go:64] FLAG: --image-service-endpoint=""
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904485    2548 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904488    2548 flags.go:64] FLAG: --kube-api-burst="100"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904491    2548 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904494    2548 flags.go:64] FLAG: --kube-api-qps="50"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904497    2548 flags.go:64] FLAG: --kube-reserved=""
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904500    2548 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904502    2548 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904505    2548 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904508    2548 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904510    2548 flags.go:64] FLAG: --lock-file=""
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904513    2548 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 27 15:05:28.911172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904516    2548 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904519    2548 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904524    2548 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904527    2548 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904550    2548 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904552    2548 flags.go:64] FLAG: --logging-format="text"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904556    2548 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904559    2548 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904565    2548 flags.go:64] FLAG: --manifest-url=""
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904568    2548 flags.go:64] FLAG: --manifest-url-header=""
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904572    2548 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904575    2548 flags.go:64] FLAG: --max-open-files="1000000"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904579    2548 flags.go:64] FLAG: --max-pods="110"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904582    2548 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904585    2548 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904587    2548 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904590    2548 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904593    2548 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904595    2548 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904598    2548 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904605    2548 flags.go:64] FLAG: --node-status-max-images="50"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904608    2548 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904611    2548 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 27 15:05:28.911850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904614    2548 flags.go:64] FLAG: --pod-cidr=""
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904617    2548 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904622    2548 flags.go:64] FLAG: --pod-manifest-path=""
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904625    2548 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904628    2548 flags.go:64] FLAG: --pods-per-core="0"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904630    2548 flags.go:64] FLAG: --port="10250"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904633    2548 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904636    2548 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b5225a656ea1312e"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904639    2548 flags.go:64] FLAG: --qos-reserved=""
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904642    2548 flags.go:64] FLAG: --read-only-port="10255"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904645    2548 flags.go:64] FLAG: --register-node="true"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904648    2548 flags.go:64] FLAG: --register-with-taints=""
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904652    2548 flags.go:64] FLAG: --registry-burst="10"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904655    2548 flags.go:64] FLAG: --registry-qps="5"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904658    2548 flags.go:64] FLAG: --reserved-cpus=""
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904661    2548 flags.go:64] FLAG: --reserved-memory=""
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904664    2548 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904667    2548 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904671    2548 flags.go:64] FLAG: --rotate-certificates="false"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904674    2548 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904677    2548 flags.go:64] FLAG: --runonce="false"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904679    2548 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904682    2548 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904685    2548 flags.go:64] FLAG: --seccomp-default="false"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904688    2548 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 27 15:05:28.912419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904690    2548 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904693    2548 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904696    2548 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904699    2548 flags.go:64] FLAG: --storage-driver-password="root"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904702    2548 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904705    2548 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904707    2548 flags.go:64] FLAG: --storage-driver-user="root"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904710    2548 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904713    2548 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904716    2548 flags.go:64] FLAG: --system-cgroups=""
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904719    2548 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904724    2548 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904727    2548 flags.go:64] FLAG: --tls-cert-file=""
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904729    2548 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904734    2548 flags.go:64] FLAG: --tls-min-version=""
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904736    2548 flags.go:64] FLAG: --tls-private-key-file=""
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904739    2548 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904742    2548 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904745    2548 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904747    2548 flags.go:64] FLAG: --v="2"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904751    2548 flags.go:64] FLAG: --version="false"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904761    2548 flags.go:64] FLAG: --vmodule=""
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904765    2548 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.904768    2548 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904896    2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 15:05:28.913052 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904902    2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904905    2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904908    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904910    2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904913    2548 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904916    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904919    2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904921    2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904924    2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904926    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904929    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904931    2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904934    2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904937    2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904940    2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904943    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904945    2548 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904948    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904950    2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904952    2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 15:05:28.913665 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904955    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904957    2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904960    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904963    2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904965    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904968    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904970    2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904973    2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904975    2548 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904978    2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904980    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904983    2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904987    2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904990    2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904992    2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904995    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.904997    2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905000    2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905002    2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905005    2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 15:05:28.914159 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905008    2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905010    2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905013    2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905016    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905018    2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905020    2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905023    2548 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905025    2548 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905028    2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905030    2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905033    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905035    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905038    2548 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905040    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905042    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905046    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905049    2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905051    2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905053    2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905057    2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 15:05:28.914770 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905061    2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905063    2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905066    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905068    2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905072    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905075    2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905077    2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905080    2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905082    2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905085    2548 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905087    2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905090    2548 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905092    2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905095    2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905097    2548 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905100    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905103    2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905107    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905109    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905112    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 15:05:28.915266 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905115    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905117    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905119    2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905122    2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905125    2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905127    2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905129    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905133    2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905136    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905138    2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905141    2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905143    2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905146    2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905148    2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905150    2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905154    2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905159    2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 15:05:28.915791 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.905162    2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.905989    2548 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.912795    2548 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.912810    2548 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.912830    2548 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.912836    2548 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912900    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912905    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912909    2548 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912912    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912915    2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912918    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912921    2548 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912923    2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912926    2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 15:05:28.916220 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912930    2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912932    2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912935    2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912939    2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912942    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912945    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912948    2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912950    2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912953    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912955    2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912958    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912960    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912962    2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912965    2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912967    2548 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912970    2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912973    2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912975    2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912977    2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 15:05:28.916632 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912980    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912984    2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912986    2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912989    2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912992    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912994    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912997    2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.912999    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913002    2548 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913005    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913007    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913009    2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913012    2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913015    2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913017    2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913020    2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913022    2548 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913025    2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913027    2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913030    2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 15:05:28.917142 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913033    2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913036    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913038    2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913041    2548 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913043    2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913045    2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913048    2548 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913050    2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913053    2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913056    2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913058    2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913062    2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913066    2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913069    2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913073    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913076    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913079    2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913082    2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913085    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 15:05:28.917677 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913088    2548 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913092    2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913095    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913098    2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913101    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913104    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913106    2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913108    2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913111    2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913113    2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913116    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913118    2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913120    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913123    2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913126    2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913128    2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913130    2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913133    2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913135    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 15:05:28.918168 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913138    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913140    2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913142    2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913145    2548 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913147    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913150    2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913152    2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913155    2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913158    2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913161    2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913163    2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913165    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913168    2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.913173    2548 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913294    2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 15:05:28.918950 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913299    2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913302    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913305    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913307    2548 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913310    2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913313    2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913316    2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913318    2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913321    2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913324    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913326    2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913329    2548 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913331    2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913333    2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913336    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913339    2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913343    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913346    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913349    2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 15:05:28.919477 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913351    2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913354    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913356    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913359    2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913361    2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913364    2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913366    2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913369    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913372    2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913374    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913377    2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913379    2548 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913381    2548 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913384    2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913387    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913389    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913391    2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913394    2548 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913396    2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913398    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 15:05:28.920044 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913401    2548 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913403    2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913405    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913408    2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913410    2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913413    2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913415    2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913417    2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913420    2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913424    2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913427    2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913430    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913434    2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913437    2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913439    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913442    2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913445    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913448    2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913450    2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 15:05:28.920561 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913452    2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913455    2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913458    2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913460    2548 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913463    2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913465    2548 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913468    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913470    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913472    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913475    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913477    2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913480    2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913482    2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913484    2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913487    2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913489    2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913492    2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913494    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913496    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913499    2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 15:05:28.921181 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913502    2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913504    2548 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913506    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913509    2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913511    2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913514    2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913516    2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913519    2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913522    2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913525    2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913528    2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913549    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913552    2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913555    2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913557    2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913560    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913563    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913566    2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913568    2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 15:05:28.921891 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:28.913571    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 15:05:28.922467 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.913576    2548 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 15:05:28.922467 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.913682    2548 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 27 15:05:28.922467 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.916106    2548 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 27 15:05:28.922467 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.918095    2548 server.go:1019] "Starting client certificate rotation"
Jul 27 15:05:28.922467 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.918214    2548 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 15:05:28.922467 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.918265    2548 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 15:05:28.922467 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.921230    2548 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 15:05:28.922681 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.922486    2548 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 15:05:28.938748 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.938733    2548 log.go:25] "Validated CRI v1 runtime API"
Jul 27 15:05:28.940781 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.940765    2548 log.go:25] "Validated CRI v1 image API"
Jul 27 15:05:28.941565 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.941546    2548 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 27 15:05:28.944218 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.944196    2548 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 e185c14e-bbd1-486a-b769-9d9fa31e07ee:/dev/nvme0n1p4 fbe7b3fa-489e-4b9b-8b7a-8944a38831d9:/dev/nvme0n1p3]
Jul 27 15:05:28.944266 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.944222    2548 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 27 15:05:28.950688 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.950581    2548 manager.go:216] Machine: {Timestamp:2026-07-27 15:05:28.948235847 +0000 UTC m=+0.376911021 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098055 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f750e029cc7297cd7f64aa4053d17 SystemUUID:ec2f750e-029c-c729-7cd7-f64aa4053d17 BootID:20e293b8-dd9a-43bf-b36c-ed7dff637a40 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:60:28:41:0f:31 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:60:28:41:0f:31 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:d2:f8:7e:ca:14 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 27 15:05:28.950688 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.950687    2548 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 27 15:05:28.950801 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.950766    2548 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 27 15:05:28.953827 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.953799    2548 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 27 15:05:28.953972 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.953831    2548 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-46.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 27 15:05:28.954559 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.954549    2548 topology_manager.go:138] "Creating topology manager with none policy"
Jul 27 15:05:28.954591 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.954562    2548 container_manager_linux.go:309] "Creating device plugin manager"
Jul 27 15:05:28.954591 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.954569    2548 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 15:05:28.955286 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.955276    2548 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 15:05:28.956018 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.956008    2548 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 27 15:05:28.956604 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.956587    2548 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 15:05:28.958521 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.958509    2548 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 15:05:28.958727 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.958717    2548 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 27 15:05:28.958776 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.958749    2548 kubelet.go:486] "Attempting to sync node with API server"
Jul 27 15:05:28.958776 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.958759    2548 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 27 15:05:28.958838 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.958780    2548 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 27 15:05:28.958838 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.958797    2548 kubelet.go:398] "Adding apiserver pod source"
Jul 27 15:05:28.958838 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.958817    2548 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 27 15:05:28.959227 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.959216    2548 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 15:05:28.959264 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.959238    2548 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 15:05:28.962297 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.962284    2548 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 27 15:05:28.964724 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.964711    2548 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 27 15:05:28.964767 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.964742    2548 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 27 15:05:28.967068 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.967055    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 27 15:05:28.967103 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.967077    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 27 15:05:28.967103 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.967084    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 27 15:05:28.967103 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.967089    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 27 15:05:28.967103 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.967098    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 27 15:05:28.967103 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.967103    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 27 15:05:28.967231 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.967111    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 27 15:05:28.967231 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.967116    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 27 15:05:28.967231 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.967125    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 27 15:05:28.967231 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.967131    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 27 15:05:28.967231 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.967139    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 27 15:05:28.967231 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.967144    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 27 15:05:28.967916 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.967907    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 27 15:05:28.967916 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.967916    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 27 15:05:28.971139 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.971126    2548 server.go:1268] "Started kubelet"
Jul 27 15:05:28.971512 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:28.971380    2548 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 15:05:28.971512 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.971413    2548 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 27 15:05:28.977200 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:28.971382    2548 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-46.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 15:05:28.977451 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.977427    2548 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-46.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 15:05:28.977518 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.977479    2548 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6g7lf"
Jul 27 15:05:28.977882 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.977833    2548 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 27 15:05:28.977957 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.977900    2548 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 27 15:05:28.978714 ip-10-0-143-46 systemd[1]: Started Kubernetes Kubelet.
Jul 27 15:05:28.978922 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:28.978860    2548 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 27 15:05:28.979842 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.979723    2548 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 27 15:05:28.980835 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.980816    2548 server.go:310] "Adding debug handlers to kubelet server"
Jul 27 15:05:28.984072 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.984056    2548 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6g7lf"
Jul 27 15:05:28.984231 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:28.983031    2548 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-46.ec2.internal.18c62ddcdfe0f2ff  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-46.ec2.internal,UID:ip-10-0-143-46.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-46.ec2.internal,},FirstTimestamp:2026-07-27 15:05:28.971096831 +0000 UTC m=+0.399772005,LastTimestamp:2026-07-27 15:05:28.971096831 +0000 UTC m=+0.399772005,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-46.ec2.internal,}"
Jul 27 15:05:28.988502 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.988485    2548 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 27 15:05:28.988587 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.988499    2548 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 27 15:05:28.988637 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:28.988603    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-46.ec2.internal\" not found"
Jul 27 15:05:28.989399 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.989382    2548 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 27 15:05:28.989399 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.989392    2548 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 27 15:05:28.989582 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.989413    2548 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 27 15:05:28.989582 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.989506    2548 reconstruct.go:97] "Volume reconstruction finished"
Jul 27 15:05:28.989582 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.989513    2548 reconciler.go:29] "Reconciler: start to sync state"
Jul 27 15:05:28.989728 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.989610    2548 factory.go:153] Registering CRI-O factory
Jul 27 15:05:28.989728 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.989626    2548 factory.go:223] Registration of the crio container factory successfully
Jul 27 15:05:28.989728 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.989674    2548 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 27 15:05:28.989728 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.989684    2548 factory.go:55] Registering systemd factory
Jul 27 15:05:28.989728 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.989693    2548 factory.go:223] Registration of the systemd container factory successfully
Jul 27 15:05:28.989728 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.989715    2548 factory.go:103] Registering Raw factory
Jul 27 15:05:28.989728 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.989728    2548 manager.go:1196] Started watching for new ooms in manager
Jul 27 15:05:28.990144 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:28.990129    2548 manager.go:349] Starting recovery of all containers
Jul 27 15:05:29.000486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.000469    2548 manager.go:354] Recovery completed
Jul 27 15:05:29.001979 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.001952    2548 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 27 15:05:29.002424 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.002407    2548 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 15:05:29.004867 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.004851    2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 15:05:29.005717 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.005700    2548 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-46.ec2.internal\" not found" node="ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.006753 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.006737    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 15:05:29.006846 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.006773    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 15:05:29.006846 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.006788    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasSufficientPID"
Jul 27 15:05:29.007311 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.007294    2548 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 27 15:05:29.007311 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.007308    2548 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 27 15:05:29.007411 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.007320    2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 15:05:29.007411 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.007324    2548 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 15:05:29.008020 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.008007    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 15:05:29.008074 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.008036    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 15:05:29.008074 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.008051    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasSufficientPID"
Jul 27 15:05:29.008919 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.008907    2548 policy_none.go:49] "None policy: Start"
Jul 27 15:05:29.008951 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.008925    2548 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 27 15:05:29.008951 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.008935    2548 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 27 15:05:29.009960 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.009950    2548 policy_none.go:47] "Start"
Jul 27 15:05:29.057272 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.057092    2548 manager.go:337] "Starting Device Plugin manager"
Jul 27 15:05:29.057359 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.057310    2548 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 27 15:05:29.057359 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.057322    2548 server.go:87] "Starting device plugin registration server"
Jul 27 15:05:29.057547 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.057520    2548 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 27 15:05:29.057580 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.057551    2548 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 27 15:05:29.057669 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.057652    2548 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 27 15:05:29.057756 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.057735    2548 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 27 15:05:29.057756 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.057751    2548 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 27 15:05:29.058236 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.058217    2548 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 27 15:05:29.058314 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.058269    2548 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-46.ec2.internal\" not found"
Jul 27 15:05:29.130852 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.130822    2548 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 27 15:05:29.132139 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.132123    2548 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 27 15:05:29.132197 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.132162    2548 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 27 15:05:29.132197 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.132184    2548 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 27 15:05:29.132285 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.132217    2548 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 27 15:05:29.135017 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.134992    2548 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 15:05:29.158365 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.158289    2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 15:05:29.159393 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.159375    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 15:05:29.159480 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.159410    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 15:05:29.159480 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.159423    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasSufficientPID"
Jul 27 15:05:29.159480 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.159445    2548 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.166026 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.166011    2548 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.166094 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.166035    2548 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-46.ec2.internal\": node \"ip-10-0-143-46.ec2.internal\" not found"
Jul 27 15:05:29.177884 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.177862    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-46.ec2.internal\" not found"
Jul 27 15:05:29.233080 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.233050    2548 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-46.ec2.internal"]
Jul 27 15:05:29.233163 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.233123    2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 15:05:29.234047 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.234032    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 15:05:29.234114 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.234065    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 15:05:29.234114 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.234080    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasSufficientPID"
Jul 27 15:05:29.235378 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.235366    2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 15:05:29.235511 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.235495    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.235571 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.235562    2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 15:05:29.236060 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.236038    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 15:05:29.236060 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.236058    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 15:05:29.236172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.236067    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 15:05:29.236172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.236076    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 15:05:29.236172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.236082    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasSufficientPID"
Jul 27 15:05:29.236172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.236095    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasSufficientPID"
Jul 27 15:05:29.237404 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.237389    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.237450 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.237414    2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 15:05:29.238012 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.237997    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 15:05:29.238065 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.238019    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 15:05:29.238065 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.238030    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeHasSufficientPID"
Jul 27 15:05:29.263248 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.263226    2548 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-46.ec2.internal\" not found" node="ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.267695 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.267680    2548 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-46.ec2.internal\" not found" node="ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.278844 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.278827    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-46.ec2.internal\" not found"
Jul 27 15:05:29.379788 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.379747    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-46.ec2.internal\" not found"
Jul 27 15:05:29.391117 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.391096    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c7c4cbd4197f801582237d506698c93-config\") pod \"kube-apiserver-proxy-ip-10-0-143-46.ec2.internal\" (UID: \"7c7c4cbd4197f801582237d506698c93\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.391177 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.391123    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a5ee5e075f836380cf597598411a56b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal\" (UID: \"9a5ee5e075f836380cf597598411a56b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.391177 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.391142    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a5ee5e075f836380cf597598411a56b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal\" (UID: \"9a5ee5e075f836380cf597598411a56b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.480615 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.480511    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-46.ec2.internal\" not found"
Jul 27 15:05:29.491840 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.491818    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a5ee5e075f836380cf597598411a56b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal\" (UID: \"9a5ee5e075f836380cf597598411a56b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.491899 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.491883    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a5ee5e075f836380cf597598411a56b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal\" (UID: \"9a5ee5e075f836380cf597598411a56b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.491933 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.491913    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a5ee5e075f836380cf597598411a56b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal\" (UID: \"9a5ee5e075f836380cf597598411a56b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.491970 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.491943    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c7c4cbd4197f801582237d506698c93-config\") pod \"kube-apiserver-proxy-ip-10-0-143-46.ec2.internal\" (UID: \"7c7c4cbd4197f801582237d506698c93\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.491970 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.491947    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a5ee5e075f836380cf597598411a56b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal\" (UID: \"9a5ee5e075f836380cf597598411a56b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.492031 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.491978    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c7c4cbd4197f801582237d506698c93-config\") pod \"kube-apiserver-proxy-ip-10-0-143-46.ec2.internal\" (UID: \"7c7c4cbd4197f801582237d506698c93\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.566027 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.565985    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.570845 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.570825    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:29.581106 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.581091    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-46.ec2.internal\" not found"
Jul 27 15:05:29.681690 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.681654    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-46.ec2.internal\" not found"
Jul 27 15:05:29.782293 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.782210    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-46.ec2.internal\" not found"
Jul 27 15:05:29.882729 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.882681    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-46.ec2.internal\" not found"
Jul 27 15:05:29.918348 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.918306    2548 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 27 15:05:29.918854 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.918464    2548 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 15:05:29.918854 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.918499    2548 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 15:05:29.967958 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.967932    2548 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 15:05:29.983626 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:29.983567    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-46.ec2.internal\" not found"
Jul 27 15:05:29.988732 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.988702    2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 15:00:29 +0000 UTC" deadline="2027-12-24 22:24:31.381938024 +0000 UTC"
Jul 27 15:05:29.988732 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:29.988731    2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12367h19m1.393210656s"
Jul 27 15:05:30.083816 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:30.083731    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-46.ec2.internal\" not found"
Jul 27 15:05:30.184867 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:30.184838    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-46.ec2.internal\" not found"
Jul 27 15:05:30.279535 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.279512    2548 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 15:05:30.286519 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:30.286482    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a5ee5e075f836380cf597598411a56b.slice/crio-b3c1666d74f10ce1603b71fbf3ad8fe0122f98c7bc2b76844633be2a40492da5 WatchSource:0}: Error finding container b3c1666d74f10ce1603b71fbf3ad8fe0122f98c7bc2b76844633be2a40492da5: Status 404 returned error can't find the container with id b3c1666d74f10ce1603b71fbf3ad8fe0122f98c7bc2b76844633be2a40492da5
Jul 27 15:05:30.286832 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:30.286806    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c7c4cbd4197f801582237d506698c93.slice/crio-be8d89b864ed7129b82c2500ad69bd93ad35d32b7d14ded8abc059b16762e2ff WatchSource:0}: Error finding container be8d89b864ed7129b82c2500ad69bd93ad35d32b7d14ded8abc059b16762e2ff: Status 404 returned error can't find the container with id be8d89b864ed7129b82c2500ad69bd93ad35d32b7d14ded8abc059b16762e2ff
Jul 27 15:05:30.289718 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.289680    2548 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:30.293337 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.293313    2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 15:05:30.304599 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.304582    2548 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 15:05:30.305763 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.305750    2548 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-46.ec2.internal"
Jul 27 15:05:30.314691 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.314676    2548 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 15:05:30.950243 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.950211    2548 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 15:05:30.960092 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.960073    2548 apiserver.go:52] "Watching apiserver"
Jul 27 15:05:30.968865 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.968838    2548 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 15:05:30.971429 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.971123    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-bgwms","openshift-multus/multus-additional-cni-plugins-7z4g4","openshift-multus/multus-gcq99","openshift-network-diagnostics/network-check-target-r4tc2","kube-system/konnectivity-agent-8hhwl","kube-system/kube-apiserver-proxy-ip-10-0-143-46.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal","openshift-multus/network-metrics-daemon-x69pt","openshift-network-operator/iptables-alerter-t7xlb","openshift-ovn-kubernetes/ovnkube-node-j94kd","kube-system/global-pull-secret-syncer-njr4n","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp","openshift-cluster-node-tuning-operator/tuned-7txrp","openshift-dns/node-resolver-52jxv"]
Jul 27 15:05:30.972535 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.972519    2548 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 15:05:30.974045 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.974016    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:30.975219 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.975200    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:30.976376 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.976361    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gcq99"
Jul 27 15:05:30.976939 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.976920    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 15:05:30.977703 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.977682    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:30.977801 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:30.977780    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:30.978938 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.978764    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 15:05:30.978938 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.978774    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 15:05:30.978938 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.978788    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 15:05:30.978938 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.978790    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 15:05:30.978938 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.978825    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 15:05:30.978938 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.978827    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rwmd4\""
Jul 27 15:05:30.979308 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.978966    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 15:05:30.979308 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.979195    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8hhwl"
Jul 27 15:05:30.979308 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.979240    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gphkj\""
Jul 27 15:05:30.979461 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.979351    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 15:05:30.980327 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.980310    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 15:05:30.980461 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.980341    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-chjkj\""
Jul 27 15:05:30.980461 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.980363    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 15:05:30.980654 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.980384    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 15:05:30.980654 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.980386    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 15:05:30.980965 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.980944    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-52jxv"
Jul 27 15:05:30.982289 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.982027    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 15:05:30.982289 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.982114    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 15:05:30.982289 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.982166    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-plp5z\""
Jul 27 15:05:30.982584 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.982567    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t7xlb"
Jul 27 15:05:30.983233 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.983216    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 15:05:30.983576 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.983563    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mnxpm\""
Jul 27 15:05:30.983648 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.983602    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 15:05:30.983997 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.983978    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:30.984098 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:30.984061    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:30.985339 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.985322    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gbpvp\""
Jul 27 15:05:30.985434 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.985343    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 15:05:30.985434 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.985387    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 15:05:30.985434 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.985393    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 15:05:30.985872 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.985850    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:30.987288 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.987271    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bgwms"
Jul 27 15:05:30.988192 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.988173    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 15:05:30.988408 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.988391    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 15:05:30.988804 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.988764    2548 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 27 15:05:30.988804 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.988789    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g4xvv\""
Jul 27 15:05:30.989365 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.989347    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:30.989534 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:30.989511    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:30.989716 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.989698    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 15:05:30.989926 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.989904    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 15:05:30.990029 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.989946    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 15:05:30.990130 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.990111    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vwsgk\""
Jul 27 15:05:30.991976 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.991952    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:30.994205 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.994183    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 27 15:05:30.994384 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.994364    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sz6gt\""
Jul 27 15:05:30.994896 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.994871    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 15:05:30.995250 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:30.995079    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 15:05:31.001872 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.001855    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/63d99334-3633-4f35-9097-385b6aed074b-agent-certs\") pod \"konnectivity-agent-8hhwl\" (UID: \"63d99334-3633-4f35-9097-385b6aed074b\") " pod="kube-system/konnectivity-agent-8hhwl"
Jul 27 15:05:31.001981 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.001880    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs\") pod \"network-metrics-daemon-x69pt\" (UID: \"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3\") " pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:31.001981 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.001907    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92215e75-de40-412c-85a4-0eabf00f5dc1-os-release\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.001981 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.001932    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3253118-4914-4dc8-a105-d30a1f850364-host\") pod \"node-ca-bgwms\" (UID: \"c3253118-4914-4dc8-a105-d30a1f850364\") " pod="openshift-image-registry/node-ca-bgwms"
Jul 27 15:05:31.001981 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.001962    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3253118-4914-4dc8-a105-d30a1f850364-serviceca\") pod \"node-ca-bgwms\" (UID: \"c3253118-4914-4dc8-a105-d30a1f850364\") " pod="openshift-image-registry/node-ca-bgwms"
Jul 27 15:05:31.002155 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002006    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-kubelet\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.002155 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002058    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-var-lib-cni-multus\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.002155 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002085    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-multus-conf-dir\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.002155 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002100    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-registration-dir\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.002155 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002116    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hnvx\" (UniqueName: \"kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx\") pod \"network-check-target-r4tc2\" (UID: \"1e7a0804-791d-4fa4-8155-e7830a048c07\") " pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:31.002155 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002131    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-run-netns\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.002155 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002152    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.002507 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002179    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-tmp\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.002507 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002203    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dctj6\" (UniqueName: \"kubernetes.io/projected/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-kube-api-access-dctj6\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.002507 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002230    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8af6f3dc-8996-4c73-bfed-716386b74702-host-slash\") pod \"iptables-alerter-t7xlb\" (UID: \"8af6f3dc-8996-4c73-bfed-716386b74702\") " pod="openshift-network-operator/iptables-alerter-t7xlb"
Jul 27 15:05:31.002507 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002265    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-systemd\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.002507 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002345    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-run-netns\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.002507 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002391    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gvn2\" (UniqueName: \"kubernetes.io/projected/78b76a57-4e6c-4189-9225-d4e032894ca2-kube-api-access-6gvn2\") pod \"node-resolver-52jxv\" (UID: \"78b76a57-4e6c-4189-9225-d4e032894ca2\") " pod="openshift-dns/node-resolver-52jxv"
Jul 27 15:05:31.002507 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002454    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs8gh\" (UniqueName: \"kubernetes.io/projected/8af6f3dc-8996-4c73-bfed-716386b74702-kube-api-access-qs8gh\") pod \"iptables-alerter-t7xlb\" (UID: \"8af6f3dc-8996-4c73-bfed-716386b74702\") " pod="openshift-network-operator/iptables-alerter-t7xlb"
Jul 27 15:05:31.002507 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002504    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92215e75-de40-412c-85a4-0eabf00f5dc1-system-cni-dir\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.002929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002530    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-run-openvswitch\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.002929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002577    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-node-log\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.002929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002609    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-cni-netd\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.002929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002639    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.002929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002664    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/63d99334-3633-4f35-9097-385b6aed074b-konnectivity-ca\") pod \"konnectivity-agent-8hhwl\" (UID: \"63d99334-3633-4f35-9097-385b6aed074b\") " pod="kube-system/konnectivity-agent-8hhwl"
Jul 27 15:05:31.002929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002706    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92215e75-de40-412c-85a4-0eabf00f5dc1-cni-binary-copy\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.002929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002741    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-run-ovn-kubernetes\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.002929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002764    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cfdaf978-f52a-487c-b5b4-d7627b36fed9-env-overrides\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.002929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002787    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxzb8\" (UniqueName: \"kubernetes.io/projected/c3253118-4914-4dc8-a105-d30a1f850364-kube-api-access-sxzb8\") pod \"node-ca-bgwms\" (UID: \"c3253118-4914-4dc8-a105-d30a1f850364\") " pod="openshift-image-registry/node-ca-bgwms"
Jul 27 15:05:31.002929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002812    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-run-ovn\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.002929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002840    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-run\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.002929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002865    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:31.002929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002898    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92215e75-de40-412c-85a4-0eabf00f5dc1-cnibin\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.002929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002933    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92215e75-de40-412c-85a4-0eabf00f5dc1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002950    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86jl2\" (UniqueName: \"kubernetes.io/projected/cfdaf978-f52a-487c-b5b4-d7627b36fed9-kube-api-access-86jl2\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002967    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-sysctl-conf\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.002988    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-host\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003008    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-system-cni-dir\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003023    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-os-release\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003038    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f1023b4e-580a-4dc0-8a69-4167ae7cec34-kubelet-config\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003053    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-sys-fs\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003070    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfdaf978-f52a-487c-b5b4-d7627b36fed9-ovn-node-metrics-cert\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003097    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cfdaf978-f52a-487c-b5b4-d7627b36fed9-ovnkube-script-lib\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003119    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-kubernetes\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003140    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-lib-modules\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003176    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwml8\" (UniqueName: \"kubernetes.io/projected/6c6a1ca4-48f0-4dee-b65c-7833848200a9-kube-api-access-hwml8\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003209    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8af6f3dc-8996-4c73-bfed-716386b74702-iptables-alerter-script\") pod \"iptables-alerter-t7xlb\" (UID: \"8af6f3dc-8996-4c73-bfed-716386b74702\") " pod="openshift-network-operator/iptables-alerter-t7xlb"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003250    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjrr4\" (UniqueName: \"kubernetes.io/projected/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-kube-api-access-hjrr4\") pod \"network-metrics-daemon-x69pt\" (UID: \"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3\") " pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003277    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-systemd-units\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.003486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003294    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-modprobe-d\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003311    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-var-lib-openvswitch\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003324    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cfdaf978-f52a-487c-b5b4-d7627b36fed9-ovnkube-config\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003350    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-sys\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003377    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-var-lib-cni-bin\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003391    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-hostroot\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003417    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-log-socket\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003448    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-sysconfig\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003464    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-tuned\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003478    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm7gs\" (UniqueName: \"kubernetes.io/projected/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-kube-api-access-mm7gs\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003498    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-run-k8s-cni-cncf-io\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003600    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-etc-kubernetes\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003640    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jghx2\" (UniqueName: \"kubernetes.io/projected/92215e75-de40-412c-85a4-0eabf00f5dc1-kube-api-access-jghx2\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003668    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-cni-bin\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003692    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-sysctl-d\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003715    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-var-lib-kubelet\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003740    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-cni-binary-copy\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.004267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003789    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-multus-socket-dir-parent\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003835    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78b76a57-4e6c-4189-9225-d4e032894ca2-hosts-file\") pod \"node-resolver-52jxv\" (UID: \"78b76a57-4e6c-4189-9225-d4e032894ca2\") " pod="openshift-dns/node-resolver-52jxv"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003872    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92215e75-de40-412c-85a4-0eabf00f5dc1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003915    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-slash\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003942    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-run-systemd\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003967    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-etc-openvswitch\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003983    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f1023b4e-580a-4dc0-8a69-4167ae7cec34-dbus\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.003998    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-device-dir\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.004018    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92215e75-de40-412c-85a4-0eabf00f5dc1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.004079    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-multus-cni-dir\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.004109    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-cnibin\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.004132    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-var-lib-kubelet\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.004157    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-run-multus-certs\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.004181    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-socket-dir\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.004211    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78b76a57-4e6c-4189-9225-d4e032894ca2-tmp-dir\") pod \"node-resolver-52jxv\" (UID: \"78b76a57-4e6c-4189-9225-d4e032894ca2\") " pod="openshift-dns/node-resolver-52jxv"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.004245    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-multus-daemon-config\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.004962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.004302    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.006435 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.006415    2548 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 15:05:31.029671 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.029637    2548 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9cwhv"
Jul 27 15:05:31.037670 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.037655    2548 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9cwhv"
Jul 27 15:05:31.090021 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.089999    2548 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 27 15:05:31.104522 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104495    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:31.104689 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104561    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92215e75-de40-412c-85a4-0eabf00f5dc1-cnibin\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.104689 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104589    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92215e75-de40-412c-85a4-0eabf00f5dc1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.104689 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104614    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86jl2\" (UniqueName: \"kubernetes.io/projected/cfdaf978-f52a-487c-b5b4-d7627b36fed9-kube-api-access-86jl2\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.104689 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104638    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-sysctl-conf\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.104689 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104660    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-host\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.104689 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104685    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-system-cni-dir\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.104662    2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104710    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-os-release\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104745    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f1023b4e-580a-4dc0-8a69-4167ae7cec34-kubelet-config\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104756    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-host\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104770    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-sys-fs\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104775    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-system-cni-dir\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104710    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92215e75-de40-412c-85a4-0eabf00f5dc1-cnibin\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.104842    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret podName:f1023b4e-580a-4dc0-8a69-4167ae7cec34 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:31.604776749 +0000 UTC m=+3.033451911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret") pod "global-pull-secret-syncer-njr4n" (UID: "f1023b4e-580a-4dc0-8a69-4167ae7cec34") : object "kube-system"/"original-pull-secret" not registered
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104844    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-sysctl-conf\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104855    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f1023b4e-580a-4dc0-8a69-4167ae7cec34-kubelet-config\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104857    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-os-release\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104839    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-sys-fs\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104886    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfdaf978-f52a-487c-b5b4-d7627b36fed9-ovn-node-metrics-cert\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104921    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cfdaf978-f52a-487c-b5b4-d7627b36fed9-ovnkube-script-lib\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104948    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-kubernetes\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.104980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104980    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-lib-modules\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.104998    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwml8\" (UniqueName: \"kubernetes.io/projected/6c6a1ca4-48f0-4dee-b65c-7833848200a9-kube-api-access-hwml8\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105027    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8af6f3dc-8996-4c73-bfed-716386b74702-iptables-alerter-script\") pod \"iptables-alerter-t7xlb\" (UID: \"8af6f3dc-8996-4c73-bfed-716386b74702\") " pod="openshift-network-operator/iptables-alerter-t7xlb"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105058    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjrr4\" (UniqueName: \"kubernetes.io/projected/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-kube-api-access-hjrr4\") pod \"network-metrics-daemon-x69pt\" (UID: \"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3\") " pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105063    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-kubernetes\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105145    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-lib-modules\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105197    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-systemd-units\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105226    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-modprobe-d\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105256    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-var-lib-openvswitch\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105282    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cfdaf978-f52a-487c-b5b4-d7627b36fed9-ovnkube-config\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105306    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-sys\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105295    2548 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105335    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-var-lib-cni-bin\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105365    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-hostroot\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105371    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-var-lib-openvswitch\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105390    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-log-socket\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105417    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-sysconfig\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105424    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-systemd-units\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.105770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105391    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92215e75-de40-412c-85a4-0eabf00f5dc1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105445    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-tuned\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105469    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm7gs\" (UniqueName: \"kubernetes.io/projected/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-kube-api-access-mm7gs\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105475    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-var-lib-cni-bin\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105531    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-modprobe-d\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105494    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-run-k8s-cni-cncf-io\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105578    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-etc-kubernetes\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105565    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-sys\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105626    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-sysconfig\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105639    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-hostroot\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105627    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jghx2\" (UniqueName: \"kubernetes.io/projected/92215e75-de40-412c-85a4-0eabf00f5dc1-kube-api-access-jghx2\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105672    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cfdaf978-f52a-487c-b5b4-d7627b36fed9-ovnkube-script-lib\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105681    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-log-socket\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105687    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-cni-bin\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105712    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8af6f3dc-8996-4c73-bfed-716386b74702-iptables-alerter-script\") pod \"iptables-alerter-t7xlb\" (UID: \"8af6f3dc-8996-4c73-bfed-716386b74702\") " pod="openshift-network-operator/iptables-alerter-t7xlb"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105743    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-cni-bin\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105864    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-sysctl-d\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105882    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-etc-kubernetes\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.106883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105890    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-var-lib-kubelet\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105916    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-cni-binary-copy\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105947    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-run-k8s-cni-cncf-io\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.105981    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-multus-socket-dir-parent\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106012    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-var-lib-kubelet\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106047    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-multus-socket-dir-parent\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106115    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78b76a57-4e6c-4189-9225-d4e032894ca2-hosts-file\") pod \"node-resolver-52jxv\" (UID: \"78b76a57-4e6c-4189-9225-d4e032894ca2\") " pod="openshift-dns/node-resolver-52jxv"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106072    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cfdaf978-f52a-487c-b5b4-d7627b36fed9-ovnkube-config\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106201    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-sysctl-d\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106210    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78b76a57-4e6c-4189-9225-d4e032894ca2-hosts-file\") pod \"node-resolver-52jxv\" (UID: \"78b76a57-4e6c-4189-9225-d4e032894ca2\") " pod="openshift-dns/node-resolver-52jxv"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106188    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92215e75-de40-412c-85a4-0eabf00f5dc1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106312    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-slash\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106373    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92215e75-de40-412c-85a4-0eabf00f5dc1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106386    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-run-systemd\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106519    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-slash\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106522    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-etc-openvswitch\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106575    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f1023b4e-580a-4dc0-8a69-4167ae7cec34-dbus\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:31.107796 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106623    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-device-dir\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106649    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92215e75-de40-412c-85a4-0eabf00f5dc1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106658    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-etc-openvswitch\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106665    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-device-dir\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106678    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-multus-cni-dir\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106693    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-run-systemd\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106748    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f1023b4e-580a-4dc0-8a69-4167ae7cec34-dbus\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106770    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-cni-binary-copy\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106807    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-cnibin\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106835    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-var-lib-kubelet\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106856    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-cnibin\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106862    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-multus-cni-dir\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106891    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-run-multus-certs\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106903    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-var-lib-kubelet\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106938    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-socket-dir\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106971    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78b76a57-4e6c-4189-9225-d4e032894ca2-tmp-dir\") pod \"node-resolver-52jxv\" (UID: \"78b76a57-4e6c-4189-9225-d4e032894ca2\") " pod="openshift-dns/node-resolver-52jxv"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106983    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-run-multus-certs\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.106994    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-multus-daemon-config\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.108631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107044    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107064    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-socket-dir\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107073    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/63d99334-3633-4f35-9097-385b6aed074b-agent-certs\") pod \"konnectivity-agent-8hhwl\" (UID: \"63d99334-3633-4f35-9097-385b6aed074b\") " pod="kube-system/konnectivity-agent-8hhwl"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107111    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs\") pod \"network-metrics-daemon-x69pt\" (UID: \"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3\") " pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107140    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92215e75-de40-412c-85a4-0eabf00f5dc1-os-release\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107240    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92215e75-de40-412c-85a4-0eabf00f5dc1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107237    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3253118-4914-4dc8-a105-d30a1f850364-host\") pod \"node-ca-bgwms\" (UID: \"c3253118-4914-4dc8-a105-d30a1f850364\") " pod="openshift-image-registry/node-ca-bgwms"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107275    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3253118-4914-4dc8-a105-d30a1f850364-serviceca\") pod \"node-ca-bgwms\" (UID: \"c3253118-4914-4dc8-a105-d30a1f850364\") " pod="openshift-image-registry/node-ca-bgwms"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107314    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107325    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-kubelet\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107352    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-var-lib-cni-multus\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107361    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3253118-4914-4dc8-a105-d30a1f850364-host\") pod \"node-ca-bgwms\" (UID: \"c3253118-4914-4dc8-a105-d30a1f850364\") " pod="openshift-image-registry/node-ca-bgwms"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107379    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92215e75-de40-412c-85a4-0eabf00f5dc1-os-release\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107317    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78b76a57-4e6c-4189-9225-d4e032894ca2-tmp-dir\") pod \"node-resolver-52jxv\" (UID: \"78b76a57-4e6c-4189-9225-d4e032894ca2\") " pod="openshift-dns/node-resolver-52jxv"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107400    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-multus-conf-dir\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107436    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-var-lib-cni-multus\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.107458    2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107486    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-kubelet\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.109380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107487    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-registration-dir\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.107505    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs podName:f21ffbbc-abb4-4ee7-8fbb-8780412fffc3 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:31.607488939 +0000 UTC m=+3.036164118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs") pod "network-metrics-daemon-x69pt" (UID: "f21ffbbc-abb4-4ee7-8fbb-8780412fffc3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107438    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-registration-dir\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107541    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hnvx\" (UniqueName: \"kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx\") pod \"network-check-target-r4tc2\" (UID: \"1e7a0804-791d-4fa4-8155-e7830a048c07\") " pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107564    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-multus-conf-dir\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107567    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-run-netns\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107626    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107640    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-run-netns\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107655    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-tmp\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107683    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dctj6\" (UniqueName: \"kubernetes.io/projected/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-kube-api-access-dctj6\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107688    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-multus-daemon-config\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107710    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8af6f3dc-8996-4c73-bfed-716386b74702-host-slash\") pod \"iptables-alerter-t7xlb\" (UID: \"8af6f3dc-8996-4c73-bfed-716386b74702\") " pod="openshift-network-operator/iptables-alerter-t7xlb"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107740    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-systemd\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107766    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-run-netns\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107794    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gvn2\" (UniqueName: \"kubernetes.io/projected/78b76a57-4e6c-4189-9225-d4e032894ca2-kube-api-access-6gvn2\") pod \"node-resolver-52jxv\" (UID: \"78b76a57-4e6c-4189-9225-d4e032894ca2\") " pod="openshift-dns/node-resolver-52jxv"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107819    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs8gh\" (UniqueName: \"kubernetes.io/projected/8af6f3dc-8996-4c73-bfed-716386b74702-kube-api-access-qs8gh\") pod \"iptables-alerter-t7xlb\" (UID: \"8af6f3dc-8996-4c73-bfed-716386b74702\") " pod="openshift-network-operator/iptables-alerter-t7xlb"
Jul 27 15:05:31.109948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107846    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92215e75-de40-412c-85a4-0eabf00f5dc1-system-cni-dir\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107871    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107874    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-run-openvswitch\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107895    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3253118-4914-4dc8-a105-d30a1f850364-serviceca\") pod \"node-ca-bgwms\" (UID: \"c3253118-4914-4dc8-a105-d30a1f850364\") " pod="openshift-image-registry/node-ca-bgwms"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107910    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-run-openvswitch\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107924    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-node-log\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107957    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-host-run-netns\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.107949    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-cni-netd\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108000    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108003    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8af6f3dc-8996-4c73-bfed-716386b74702-host-slash\") pod \"iptables-alerter-t7xlb\" (UID: \"8af6f3dc-8996-4c73-bfed-716386b74702\") " pod="openshift-network-operator/iptables-alerter-t7xlb"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108031    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/63d99334-3633-4f35-9097-385b6aed074b-konnectivity-ca\") pod \"konnectivity-agent-8hhwl\" (UID: \"63d99334-3633-4f35-9097-385b6aed074b\") " pod="kube-system/konnectivity-agent-8hhwl"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108057    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92215e75-de40-412c-85a4-0eabf00f5dc1-cni-binary-copy\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108088    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-run-ovn-kubernetes\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108107    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-systemd\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108112    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cfdaf978-f52a-487c-b5b4-d7627b36fed9-env-overrides\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108140    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxzb8\" (UniqueName: \"kubernetes.io/projected/c3253118-4914-4dc8-a105-d30a1f850364-kube-api-access-sxzb8\") pod \"node-ca-bgwms\" (UID: \"c3253118-4914-4dc8-a105-d30a1f850364\") " pod="openshift-image-registry/node-ca-bgwms"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108165    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-run-ovn\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.110437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108188    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-run\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.110988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108183    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c6a1ca4-48f0-4dee-b65c-7833848200a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.110988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108345    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-node-log\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.110988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108417    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92215e75-de40-412c-85a4-0eabf00f5dc1-system-cni-dir\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.110988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108468    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-run-ovn-kubernetes\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.110988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108481    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-run\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.110988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108503    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-host-cni-netd\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.110988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108522    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cfdaf978-f52a-487c-b5b4-d7627b36fed9-run-ovn\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.110988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.108998    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cfdaf978-f52a-487c-b5b4-d7627b36fed9-env-overrides\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.110988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.109042    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92215e75-de40-412c-85a4-0eabf00f5dc1-cni-binary-copy\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.110988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.109091    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/63d99334-3633-4f35-9097-385b6aed074b-konnectivity-ca\") pod \"konnectivity-agent-8hhwl\" (UID: \"63d99334-3633-4f35-9097-385b6aed074b\") " pod="kube-system/konnectivity-agent-8hhwl"
Jul 27 15:05:31.110988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.109407    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-etc-tuned\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.110988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.109460    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfdaf978-f52a-487c-b5b4-d7627b36fed9-ovn-node-metrics-cert\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.110988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.110004    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/63d99334-3633-4f35-9097-385b6aed074b-agent-certs\") pod \"konnectivity-agent-8hhwl\" (UID: \"63d99334-3633-4f35-9097-385b6aed074b\") " pod="kube-system/konnectivity-agent-8hhwl"
Jul 27 15:05:31.110988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.110306    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-tmp\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.127626 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.127577    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 15:05:31.127626 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.127625    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 15:05:31.127785 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.127641    2548 projected.go:196] Error preparing data for projected volume kube-api-access-7hnvx for pod openshift-network-diagnostics/network-check-target-r4tc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 15:05:31.127864 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.127832    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx podName:1e7a0804-791d-4fa4-8155-e7830a048c07 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:31.627802693 +0000 UTC m=+3.056477856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7hnvx" (UniqueName: "kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx") pod "network-check-target-r4tc2" (UID: "1e7a0804-791d-4fa4-8155-e7830a048c07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 15:05:31.128381 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.128354    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86jl2\" (UniqueName: \"kubernetes.io/projected/cfdaf978-f52a-487c-b5b4-d7627b36fed9-kube-api-access-86jl2\") pod \"ovnkube-node-j94kd\" (UID: \"cfdaf978-f52a-487c-b5b4-d7627b36fed9\") " pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.128957 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.128933    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jghx2\" (UniqueName: \"kubernetes.io/projected/92215e75-de40-412c-85a4-0eabf00f5dc1-kube-api-access-jghx2\") pod \"multus-additional-cni-plugins-7z4g4\" (UID: \"92215e75-de40-412c-85a4-0eabf00f5dc1\") " pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.130333 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.130280    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm7gs\" (UniqueName: \"kubernetes.io/projected/321a4211-4b2a-4ee9-8a0f-1d5712900bf5-kube-api-access-mm7gs\") pod \"tuned-7txrp\" (UID: \"321a4211-4b2a-4ee9-8a0f-1d5712900bf5\") " pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.130662 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.130642    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxzb8\" (UniqueName: \"kubernetes.io/projected/c3253118-4914-4dc8-a105-d30a1f850364-kube-api-access-sxzb8\") pod \"node-ca-bgwms\" (UID: \"c3253118-4914-4dc8-a105-d30a1f850364\") " pod="openshift-image-registry/node-ca-bgwms"
Jul 27 15:05:31.132579 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.132553    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjrr4\" (UniqueName: \"kubernetes.io/projected/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-kube-api-access-hjrr4\") pod \"network-metrics-daemon-x69pt\" (UID: \"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3\") " pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:31.132953 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.132931    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwml8\" (UniqueName: \"kubernetes.io/projected/6c6a1ca4-48f0-4dee-b65c-7833848200a9-kube-api-access-hwml8\") pod \"aws-ebs-csi-driver-node-zhvtp\" (UID: \"6c6a1ca4-48f0-4dee-b65c-7833848200a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.133043 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.132939    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gvn2\" (UniqueName: \"kubernetes.io/projected/78b76a57-4e6c-4189-9225-d4e032894ca2-kube-api-access-6gvn2\") pod \"node-resolver-52jxv\" (UID: \"78b76a57-4e6c-4189-9225-d4e032894ca2\") " pod="openshift-dns/node-resolver-52jxv"
Jul 27 15:05:31.133225 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.133206    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dctj6\" (UniqueName: \"kubernetes.io/projected/e9ebfe61-7a6e-4c9f-9437-0096cb7b7531-kube-api-access-dctj6\") pod \"multus-gcq99\" (UID: \"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531\") " pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.133289 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.133253    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs8gh\" (UniqueName: \"kubernetes.io/projected/8af6f3dc-8996-4c73-bfed-716386b74702-kube-api-access-qs8gh\") pod \"iptables-alerter-t7xlb\" (UID: \"8af6f3dc-8996-4c73-bfed-716386b74702\") " pod="openshift-network-operator/iptables-alerter-t7xlb"
Jul 27 15:05:31.138143 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.138100    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal" event={"ID":"9a5ee5e075f836380cf597598411a56b","Type":"ContainerStarted","Data":"b3c1666d74f10ce1603b71fbf3ad8fe0122f98c7bc2b76844633be2a40492da5"}
Jul 27 15:05:31.139272 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.139248    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-46.ec2.internal" event={"ID":"7c7c4cbd4197f801582237d506698c93","Type":"ContainerStarted","Data":"be8d89b864ed7129b82c2500ad69bd93ad35d32b7d14ded8abc059b16762e2ff"}
Jul 27 15:05:31.287143 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.287037    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:31.292625 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.292602    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7txrp"
Jul 27 15:05:31.297007 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:31.296974    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfdaf978_f52a_487c_b5b4_d7627b36fed9.slice/crio-ce621715421f48683008fe865c6368f488f9f6c1496d5ee206f498ea60276820 WatchSource:0}: Error finding container ce621715421f48683008fe865c6368f488f9f6c1496d5ee206f498ea60276820: Status 404 returned error can't find the container with id ce621715421f48683008fe865c6368f488f9f6c1496d5ee206f498ea60276820
Jul 27 15:05:31.301487 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.301464    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7z4g4"
Jul 27 15:05:31.301848 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:31.301817    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod321a4211_4b2a_4ee9_8a0f_1d5712900bf5.slice/crio-904eb6fd3abae66dc0fcc1229875821c077ebe3d6b65540406b103897a4e736b WatchSource:0}: Error finding container 904eb6fd3abae66dc0fcc1229875821c077ebe3d6b65540406b103897a4e736b: Status 404 returned error can't find the container with id 904eb6fd3abae66dc0fcc1229875821c077ebe3d6b65540406b103897a4e736b
Jul 27 15:05:31.307355 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.307330    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gcq99"
Jul 27 15:05:31.313283 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.313210    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8hhwl"
Jul 27 15:05:31.313671 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:31.313645    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92215e75_de40_412c_85a4_0eabf00f5dc1.slice/crio-cf2252debf8f88de73d3358f2bb18914a4918237bfb7a32efbf47c50300ca509 WatchSource:0}: Error finding container cf2252debf8f88de73d3358f2bb18914a4918237bfb7a32efbf47c50300ca509: Status 404 returned error can't find the container with id cf2252debf8f88de73d3358f2bb18914a4918237bfb7a32efbf47c50300ca509
Jul 27 15:05:31.315982 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:31.315874    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9ebfe61_7a6e_4c9f_9437_0096cb7b7531.slice/crio-5e578b7976a9909fbb15c2d86a9abfdc284ecc893af4bbb2c0930cf2db037ecf WatchSource:0}: Error finding container 5e578b7976a9909fbb15c2d86a9abfdc284ecc893af4bbb2c0930cf2db037ecf: Status 404 returned error can't find the container with id 5e578b7976a9909fbb15c2d86a9abfdc284ecc893af4bbb2c0930cf2db037ecf
Jul 27 15:05:31.318328 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.318306    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-52jxv"
Jul 27 15:05:31.322115 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:31.322092    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63d99334_3633_4f35_9097_385b6aed074b.slice/crio-8f3af497f6b222d5a26ed85f3829e3003340cf7f049fe4192181979ba7090661 WatchSource:0}: Error finding container 8f3af497f6b222d5a26ed85f3829e3003340cf7f049fe4192181979ba7090661: Status 404 returned error can't find the container with id 8f3af497f6b222d5a26ed85f3829e3003340cf7f049fe4192181979ba7090661
Jul 27 15:05:31.324633 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.324565    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t7xlb"
Jul 27 15:05:31.326654 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:31.326631    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78b76a57_4e6c_4189_9225_d4e032894ca2.slice/crio-4dd25689c4b9f6a179881dc1acc01eba11b41583b416b2b6b75e4875a8f8be12 WatchSource:0}: Error finding container 4dd25689c4b9f6a179881dc1acc01eba11b41583b416b2b6b75e4875a8f8be12: Status 404 returned error can't find the container with id 4dd25689c4b9f6a179881dc1acc01eba11b41583b416b2b6b75e4875a8f8be12
Jul 27 15:05:31.329140 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.329127    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bgwms"
Jul 27 15:05:31.332899 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:31.332879    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8af6f3dc_8996_4c73_bfed_716386b74702.slice/crio-dcfb6396d87c6eb7e98c25a05e49255ddfc285c3e55c31d76ef08e12b69d37e9 WatchSource:0}: Error finding container dcfb6396d87c6eb7e98c25a05e49255ddfc285c3e55c31d76ef08e12b69d37e9: Status 404 returned error can't find the container with id dcfb6396d87c6eb7e98c25a05e49255ddfc285c3e55c31d76ef08e12b69d37e9
Jul 27 15:05:31.333980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.333962    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp"
Jul 27 15:05:31.337723 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:31.337702    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3253118_4914_4dc8_a105_d30a1f850364.slice/crio-7ac07741a379d19044565d3da4324daf285d1591c0dd90d63cecc78a43f06783 WatchSource:0}: Error finding container 7ac07741a379d19044565d3da4324daf285d1591c0dd90d63cecc78a43f06783: Status 404 returned error can't find the container with id 7ac07741a379d19044565d3da4324daf285d1591c0dd90d63cecc78a43f06783
Jul 27 15:05:31.342850 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:05:31.342832    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c6a1ca4_48f0_4dee_b65c_7833848200a9.slice/crio-2dbe72f581e20ee057b7c3ddcc44ae6273c99bb95accff8a6b65be03b0ee0843 WatchSource:0}: Error finding container 2dbe72f581e20ee057b7c3ddcc44ae6273c99bb95accff8a6b65be03b0ee0843: Status 404 returned error can't find the container with id 2dbe72f581e20ee057b7c3ddcc44ae6273c99bb95accff8a6b65be03b0ee0843
Jul 27 15:05:31.611011 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.610919    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs\") pod \"network-metrics-daemon-x69pt\" (UID: \"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3\") " pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:31.611011 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.610980    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:31.611236 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.611080    2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 15:05:31.611236 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.611103    2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 15:05:31.611236 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.611143    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs podName:f21ffbbc-abb4-4ee7-8fbb-8780412fffc3 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:32.611128017 +0000 UTC m=+4.039803180 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs") pod "network-metrics-daemon-x69pt" (UID: "f21ffbbc-abb4-4ee7-8fbb-8780412fffc3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 15:05:31.611236 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.611163    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret podName:f1023b4e-580a-4dc0-8a69-4167ae7cec34 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:32.611153656 +0000 UTC m=+4.039828850 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret") pod "global-pull-secret-syncer-njr4n" (UID: "f1023b4e-580a-4dc0-8a69-4167ae7cec34") : object "kube-system"/"original-pull-secret" not registered
Jul 27 15:05:31.711429 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:31.711388    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hnvx\" (UniqueName: \"kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx\") pod \"network-check-target-r4tc2\" (UID: \"1e7a0804-791d-4fa4-8155-e7830a048c07\") " pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:31.711603 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.711556    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 15:05:31.711603 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.711577    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 15:05:31.711603 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.711589    2548 projected.go:196] Error preparing data for projected volume kube-api-access-7hnvx for pod openshift-network-diagnostics/network-check-target-r4tc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 15:05:31.711750 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:31.711663    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx podName:1e7a0804-791d-4fa4-8155-e7830a048c07 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:32.711648268 +0000 UTC m=+4.140323431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7hnvx" (UniqueName: "kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx") pod "network-check-target-r4tc2" (UID: "1e7a0804-791d-4fa4-8155-e7830a048c07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 15:05:32.040843 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:32.040788    2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 15:00:31 +0000 UTC" deadline="2028-02-03 15:06:31.826181665 +0000 UTC"
Jul 27 15:05:32.040843 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:32.040840    2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13344h0m59.785346948s"
Jul 27 15:05:32.152629 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:32.152568    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd" event={"ID":"cfdaf978-f52a-487c-b5b4-d7627b36fed9","Type":"ContainerStarted","Data":"ce621715421f48683008fe865c6368f488f9f6c1496d5ee206f498ea60276820"}
Jul 27 15:05:32.162479 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:32.162362    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp" event={"ID":"6c6a1ca4-48f0-4dee-b65c-7833848200a9","Type":"ContainerStarted","Data":"2dbe72f581e20ee057b7c3ddcc44ae6273c99bb95accff8a6b65be03b0ee0843"}
Jul 27 15:05:32.166481 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:32.166448    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t7xlb" event={"ID":"8af6f3dc-8996-4c73-bfed-716386b74702","Type":"ContainerStarted","Data":"dcfb6396d87c6eb7e98c25a05e49255ddfc285c3e55c31d76ef08e12b69d37e9"}
Jul 27 15:05:32.170441 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:32.170411    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-52jxv" event={"ID":"78b76a57-4e6c-4189-9225-d4e032894ca2","Type":"ContainerStarted","Data":"4dd25689c4b9f6a179881dc1acc01eba11b41583b416b2b6b75e4875a8f8be12"}
Jul 27 15:05:32.173301 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:32.173274    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8hhwl" event={"ID":"63d99334-3633-4f35-9097-385b6aed074b","Type":"ContainerStarted","Data":"8f3af497f6b222d5a26ed85f3829e3003340cf7f049fe4192181979ba7090661"}
Jul 27 15:05:32.177010 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:32.176982    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z4g4" event={"ID":"92215e75-de40-412c-85a4-0eabf00f5dc1","Type":"ContainerStarted","Data":"cf2252debf8f88de73d3358f2bb18914a4918237bfb7a32efbf47c50300ca509"}
Jul 27 15:05:32.180714 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:32.180680    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7txrp" event={"ID":"321a4211-4b2a-4ee9-8a0f-1d5712900bf5","Type":"ContainerStarted","Data":"904eb6fd3abae66dc0fcc1229875821c077ebe3d6b65540406b103897a4e736b"}
Jul 27 15:05:32.187200 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:32.185141    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bgwms" event={"ID":"c3253118-4914-4dc8-a105-d30a1f850364","Type":"ContainerStarted","Data":"7ac07741a379d19044565d3da4324daf285d1591c0dd90d63cecc78a43f06783"}
Jul 27 15:05:32.194681 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:32.192140    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gcq99" event={"ID":"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531","Type":"ContainerStarted","Data":"5e578b7976a9909fbb15c2d86a9abfdc284ecc893af4bbb2c0930cf2db037ecf"}
Jul 27 15:05:32.618707 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:32.618609    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:32.618887 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:32.618778    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs\") pod \"network-metrics-daemon-x69pt\" (UID: \"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3\") " pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:32.618997 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:32.618974    2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 15:05:32.619064 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:32.619043    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs podName:f21ffbbc-abb4-4ee7-8fbb-8780412fffc3 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:34.619023648 +0000 UTC m=+6.047698816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs") pod "network-metrics-daemon-x69pt" (UID: "f21ffbbc-abb4-4ee7-8fbb-8780412fffc3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 15:05:32.619596 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:32.619574    2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 15:05:32.619732 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:32.619651    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret podName:f1023b4e-580a-4dc0-8a69-4167ae7cec34 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:34.619634383 +0000 UTC m=+6.048309548 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret") pod "global-pull-secret-syncer-njr4n" (UID: "f1023b4e-580a-4dc0-8a69-4167ae7cec34") : object "kube-system"/"original-pull-secret" not registered
Jul 27 15:05:32.720311 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:32.720267    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hnvx\" (UniqueName: \"kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx\") pod \"network-check-target-r4tc2\" (UID: \"1e7a0804-791d-4fa4-8155-e7830a048c07\") " pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:32.720573 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:32.720525    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 15:05:32.720573 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:32.720552    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 15:05:32.720573 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:32.720567    2548 projected.go:196] Error preparing data for projected volume kube-api-access-7hnvx for pod openshift-network-diagnostics/network-check-target-r4tc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 15:05:32.720818 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:32.720649    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx podName:1e7a0804-791d-4fa4-8155-e7830a048c07 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:34.72062872 +0000 UTC m=+6.149303886 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7hnvx" (UniqueName: "kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx") pod "network-check-target-r4tc2" (UID: "1e7a0804-791d-4fa4-8155-e7830a048c07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 15:05:33.041356 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:33.041235    2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 15:00:31 +0000 UTC" deadline="2028-01-31 08:17:54.990576313 +0000 UTC"
Jul 27 15:05:33.041356 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:33.041301    2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13265h12m21.949279705s"
Jul 27 15:05:33.136291 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:33.135773    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:33.136291 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:33.135811    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:33.136291 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:33.135781    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:33.136291 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:33.135915    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:33.136291 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:33.135984    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:33.136291 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:33.136055    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:34.638064 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:34.637963    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:34.638630 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:34.638121    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs\") pod \"network-metrics-daemon-x69pt\" (UID: \"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3\") " pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:34.638630 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:34.638128    2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 15:05:34.638630 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:34.638200    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret podName:f1023b4e-580a-4dc0-8a69-4167ae7cec34 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:38.638179104 +0000 UTC m=+10.066854288 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret") pod "global-pull-secret-syncer-njr4n" (UID: "f1023b4e-580a-4dc0-8a69-4167ae7cec34") : object "kube-system"/"original-pull-secret" not registered
Jul 27 15:05:34.638630 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:34.638241    2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 15:05:34.638630 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:34.638278    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs podName:f21ffbbc-abb4-4ee7-8fbb-8780412fffc3 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:38.638265074 +0000 UTC m=+10.066940238 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs") pod "network-metrics-daemon-x69pt" (UID: "f21ffbbc-abb4-4ee7-8fbb-8780412fffc3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 15:05:34.739393 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:34.739343    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hnvx\" (UniqueName: \"kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx\") pod \"network-check-target-r4tc2\" (UID: \"1e7a0804-791d-4fa4-8155-e7830a048c07\") " pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:34.739625 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:34.739560    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 15:05:34.739625 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:34.739613    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 15:05:34.739625 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:34.739627    2548 projected.go:196] Error preparing data for projected volume kube-api-access-7hnvx for pod openshift-network-diagnostics/network-check-target-r4tc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 15:05:34.739820 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:34.739708    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx podName:1e7a0804-791d-4fa4-8155-e7830a048c07 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:38.739688159 +0000 UTC m=+10.168363325 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7hnvx" (UniqueName: "kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx") pod "network-check-target-r4tc2" (UID: "1e7a0804-791d-4fa4-8155-e7830a048c07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 15:05:35.136697 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:35.135505    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:35.136697 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:35.135666    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:35.136697 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:35.136085    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:35.136697 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:35.136182    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:35.136697 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:35.136224    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:35.136697 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:35.136298    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:37.132863 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:37.132775    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:37.133316 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:37.132930    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:37.133316 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:37.133085    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:37.133316 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:37.133190    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:37.133829 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:37.133797    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:37.134586 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:37.134366    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:38.675250 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:38.675209    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:38.675772 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:38.675299    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs\") pod \"network-metrics-daemon-x69pt\" (UID: \"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3\") " pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:38.675772 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:38.675389    2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 15:05:38.675772 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:38.675456    2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 15:05:38.675772 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:38.675472    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret podName:f1023b4e-580a-4dc0-8a69-4167ae7cec34 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:46.675448726 +0000 UTC m=+18.104123897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret") pod "global-pull-secret-syncer-njr4n" (UID: "f1023b4e-580a-4dc0-8a69-4167ae7cec34") : object "kube-system"/"original-pull-secret" not registered
Jul 27 15:05:38.675772 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:38.675513    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs podName:f21ffbbc-abb4-4ee7-8fbb-8780412fffc3 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:46.67549286 +0000 UTC m=+18.104168023 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs") pod "network-metrics-daemon-x69pt" (UID: "f21ffbbc-abb4-4ee7-8fbb-8780412fffc3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 15:05:38.776766 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:38.776626    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hnvx\" (UniqueName: \"kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx\") pod \"network-check-target-r4tc2\" (UID: \"1e7a0804-791d-4fa4-8155-e7830a048c07\") " pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:38.776964 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:38.776798    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 15:05:38.776964 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:38.776817    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 15:05:38.776964 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:38.776830    2548 projected.go:196] Error preparing data for projected volume kube-api-access-7hnvx for pod openshift-network-diagnostics/network-check-target-r4tc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 15:05:38.776964 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:38.776897    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx podName:1e7a0804-791d-4fa4-8155-e7830a048c07 nodeName:}" failed. No retries permitted until 2026-07-27 15:05:46.776877232 +0000 UTC m=+18.205552399 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7hnvx" (UniqueName: "kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx") pod "network-check-target-r4tc2" (UID: "1e7a0804-791d-4fa4-8155-e7830a048c07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 15:05:39.133434 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:39.133392    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:39.133662 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:39.133530    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:39.134211 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:39.133944    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:39.134211 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:39.134042    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:39.134211 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:39.134105    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:39.134211 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:39.134177    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:41.136122 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:41.136083    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:41.136528 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:41.136240    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:41.136528 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:41.136250    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:41.136528 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:41.136238    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:41.136528 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:41.136379    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:41.136528 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:41.136465    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:43.135058 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:43.135021    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:43.135444 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:43.135111    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:43.135444 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:43.135125    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:43.135444 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:43.135150    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:43.135444 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:43.135230    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:43.135444 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:43.135299    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:45.135351 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:45.135316    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:45.135812 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:45.135429    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:45.135812 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:45.135438    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:45.135812 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:45.135433    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:45.135812 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:45.135561    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:45.135812 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:45.135645    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:46.747049 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:46.747007    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:46.747556 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:46.747076    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs\") pod \"network-metrics-daemon-x69pt\" (UID: \"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3\") " pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:46.747556 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:46.747169    2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 15:05:46.747556 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:46.747203    2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 15:05:46.747556 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:46.747244    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret podName:f1023b4e-580a-4dc0-8a69-4167ae7cec34 nodeName:}" failed. No retries permitted until 2026-07-27 15:06:02.747222396 +0000 UTC m=+34.175897561 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret") pod "global-pull-secret-syncer-njr4n" (UID: "f1023b4e-580a-4dc0-8a69-4167ae7cec34") : object "kube-system"/"original-pull-secret" not registered
Jul 27 15:05:46.747556 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:46.747263    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs podName:f21ffbbc-abb4-4ee7-8fbb-8780412fffc3 nodeName:}" failed. No retries permitted until 2026-07-27 15:06:02.747256114 +0000 UTC m=+34.175931275 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs") pod "network-metrics-daemon-x69pt" (UID: "f21ffbbc-abb4-4ee7-8fbb-8780412fffc3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 15:05:46.847935 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:46.847898    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hnvx\" (UniqueName: \"kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx\") pod \"network-check-target-r4tc2\" (UID: \"1e7a0804-791d-4fa4-8155-e7830a048c07\") " pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:46.848122 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:46.848038    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 15:05:46.848122 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:46.848055    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 15:05:46.848122 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:46.848065    2548 projected.go:196] Error preparing data for projected volume kube-api-access-7hnvx for pod openshift-network-diagnostics/network-check-target-r4tc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 15:05:46.848122 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:46.848122    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx podName:1e7a0804-791d-4fa4-8155-e7830a048c07 nodeName:}" failed. No retries permitted until 2026-07-27 15:06:02.848103005 +0000 UTC m=+34.276778170 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7hnvx" (UniqueName: "kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx") pod "network-check-target-r4tc2" (UID: "1e7a0804-791d-4fa4-8155-e7830a048c07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 15:05:47.133573 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:47.133475    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:47.133573 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:47.133495    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:47.133791 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:47.133649    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:47.133791 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:47.133735    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:47.133890 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:47.133817    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:47.133942 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:47.133915    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:49.134988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:49.134952    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:49.135408 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:49.135060    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:49.135408 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:49.135088    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:49.135408 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:49.135170    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:49.135408 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:49.135061    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:49.135408 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:49.135260    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:51.132866 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:51.132826    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:51.133412 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:51.132832    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:51.133412 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:51.132970    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:51.133412 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:51.132841    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:51.133412 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:51.133055    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:51.133412 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:51.133127    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:53.132756 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.132562    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:53.133499 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.132766    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:53.133499 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:53.132870    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:53.133499 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:53.133334    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:53.133499 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.133412    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:53.133662 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:53.133502    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:53.239951 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.239921    2548 generic.go:356] "Generic (PLEG): container finished" podID="9a5ee5e075f836380cf597598411a56b" containerID="e0c18856964c8a85161ee304aba98024a8cf828218368d8f63a3bccfce07c521" exitCode=0
Jul 27 15:05:53.240144 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.240014    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal" event={"ID":"9a5ee5e075f836380cf597598411a56b","Type":"ContainerDied","Data":"e0c18856964c8a85161ee304aba98024a8cf828218368d8f63a3bccfce07c521"}
Jul 27 15:05:53.241473 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.241448    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp" event={"ID":"6c6a1ca4-48f0-4dee-b65c-7833848200a9","Type":"ContainerStarted","Data":"f69f5c6aa3fe57eaccf15811e11034a12a4787add82508f655ac3d0dea85f3e0"}
Jul 27 15:05:53.242751 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.242730    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-52jxv" event={"ID":"78b76a57-4e6c-4189-9225-d4e032894ca2","Type":"ContainerStarted","Data":"ea4197efceb3746ee913cd8c72fc19738eacd264703e05e69720a36ac422ae39"}
Jul 27 15:05:53.244240 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.244220    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8hhwl" event={"ID":"63d99334-3633-4f35-9097-385b6aed074b","Type":"ContainerStarted","Data":"1d658cab78e401edd1e9ad544be526c6d24264967476bdf4cffdfbbfd73b2c35"}
Jul 27 15:05:53.245861 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.245840    2548 generic.go:356] "Generic (PLEG): container finished" podID="92215e75-de40-412c-85a4-0eabf00f5dc1" containerID="def1cee58aa4855496dfe48ac642483212653b6a86ad68d2e26b7c54541c9960" exitCode=0
Jul 27 15:05:53.245971 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.245916    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z4g4" event={"ID":"92215e75-de40-412c-85a4-0eabf00f5dc1","Type":"ContainerDied","Data":"def1cee58aa4855496dfe48ac642483212653b6a86ad68d2e26b7c54541c9960"}
Jul 27 15:05:53.247799 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.247777    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7txrp" event={"ID":"321a4211-4b2a-4ee9-8a0f-1d5712900bf5","Type":"ContainerStarted","Data":"07736dfc714b7d7bf98eb5f455fa306b0b811f81a62c2a46e4cf5f99a63ce84d"}
Jul 27 15:05:53.249353 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.249327    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-46.ec2.internal" event={"ID":"7c7c4cbd4197f801582237d506698c93","Type":"ContainerStarted","Data":"4c3a760efc00f23283e0123a7a92f7fbedf8b3ca88ca427356ce06c966aed7cc"}
Jul 27 15:05:53.250837 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.250817    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bgwms" event={"ID":"c3253118-4914-4dc8-a105-d30a1f850364","Type":"ContainerStarted","Data":"34ceb12706586f060e8a8747f7d1a2f96c2ab9b8e34c079d3429320368c6cc01"}
Jul 27 15:05:53.252443 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.252418    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gcq99" event={"ID":"e9ebfe61-7a6e-4c9f-9437-0096cb7b7531","Type":"ContainerStarted","Data":"d62e974525271e2f77632f43e96c6290a102cd3e9319ccf3b3f16db29d13381d"}
Jul 27 15:05:53.255497 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.255476    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd" event={"ID":"cfdaf978-f52a-487c-b5b4-d7627b36fed9","Type":"ContainerStarted","Data":"f13b0490902cc40f288164acd8997fd7ff0a112d32c2e3515429e34b45e0dd10"}
Jul 27 15:05:53.255608 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.255568    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd" event={"ID":"cfdaf978-f52a-487c-b5b4-d7627b36fed9","Type":"ContainerStarted","Data":"20da1ece228b07c58ec5e5aba6b4eada130f9bb205f70abfad60b5cc332da0e2"}
Jul 27 15:05:53.255666 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.255621    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd" event={"ID":"cfdaf978-f52a-487c-b5b4-d7627b36fed9","Type":"ContainerStarted","Data":"72e7554ed8101a5283738c4bacce080afce3a355caf23f1f417830dfa16a7cc5"}
Jul 27 15:05:53.255722 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.255674    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd" event={"ID":"cfdaf978-f52a-487c-b5b4-d7627b36fed9","Type":"ContainerStarted","Data":"d08d2c6edd8b21c6816f577a3cd7a9460c073f2af936f422591855ec19f6a770"}
Jul 27 15:05:53.255778 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.255726    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd" event={"ID":"cfdaf978-f52a-487c-b5b4-d7627b36fed9","Type":"ContainerStarted","Data":"3657ea1b7b569071d461272a543ec3f59652d6cc2b1a0061e646f1d9d19fd7e1"}
Jul 27 15:05:53.255827 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.255776    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd" event={"ID":"cfdaf978-f52a-487c-b5b4-d7627b36fed9","Type":"ContainerStarted","Data":"905e6cb84b2e2769df9aeccd774bc2e251a98a499b067dc18d73bf72682dc338"}
Jul 27 15:05:53.312033 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.311975    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bgwms" podStartSLOduration=2.452984325 podStartE2EDuration="23.31195979s" totalImagesPullingTime="20.858975465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 15:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:05:53.311754256 +0000 UTC m=+24.740429439" watchObservedRunningTime="2026-07-27 15:05:53.31195979 +0000 UTC m=+24.740634974"
Jul 27 15:05:53.328010 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.327964    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-46.ec2.internal" podStartSLOduration=22.327949941 podStartE2EDuration="22.327949941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 15:05:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:05:53.327643276 +0000 UTC m=+24.756318473" watchObservedRunningTime="2026-07-27 15:05:53.327949941 +0000 UTC m=+24.756625124"
Jul 27 15:05:53.367918 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.367862    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7txrp" podStartSLOduration=2.471129363 podStartE2EDuration="23.367842755s" totalImagesPullingTime="20.896713392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 15:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:05:53.34711933 +0000 UTC m=+24.775794512" watchObservedRunningTime="2026-07-27 15:05:53.367842755 +0000 UTC m=+24.796517939"
Jul 27 15:05:53.368181 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.368150    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gcq99" podStartSLOduration=2.41833881 podStartE2EDuration="23.368140558s" totalImagesPullingTime="20.949801748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 15:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:05:53.367511089 +0000 UTC m=+24.796186279" watchObservedRunningTime="2026-07-27 15:05:53.368140558 +0000 UTC m=+24.796815746"
Jul 27 15:05:53.388839 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.388765    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-52jxv" podStartSLOduration=2.497951911 podStartE2EDuration="23.388753575s" totalImagesPullingTime="20.890801664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 15:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:05:53.388694705 +0000 UTC m=+24.817369887" watchObservedRunningTime="2026-07-27 15:05:53.388753575 +0000 UTC m=+24.817428757"
Jul 27 15:05:53.407830 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.407790    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8hhwl" podStartSLOduration=7.148666268 podStartE2EDuration="23.407778845s" totalImagesPullingTime="16.259112577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:05:53.407471318 +0000 UTC m=+24.836146502" watchObservedRunningTime="2026-07-27 15:05:53.407778845 +0000 UTC m=+24.836454027"
Jul 27 15:05:53.562688 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.562499    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:53.675737 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.675703    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:53.749363 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.749338    2548 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 27 15:05:53.843278 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.843248    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:53.921785 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:53.921714    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:54.062409 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:54.062378    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:54.062604 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:54.062451    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8hhwl"
Jul 27 15:05:54.062959 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:54.062943    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8hhwl"
Jul 27 15:05:54.076835 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:54.076744    2548 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T15:05:53.749357397Z","UUID":"dfed82b5-15b7-4897-ab0e-b3c3a7b1311d","Handler":null,"Name":"","Endpoint":""}
Jul 27 15:05:54.077962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:54.077946    2548 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 27 15:05:54.078037 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:54.077978    2548 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 27 15:05:54.260175 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:54.260137    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal" event={"ID":"9a5ee5e075f836380cf597598411a56b","Type":"ContainerStarted","Data":"a19970cb61316bd35aed1eb3d65495336856313ea2ee8d9a1f1776ea99bd738d"}
Jul 27 15:05:54.264054 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:54.262134    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp" event={"ID":"6c6a1ca4-48f0-4dee-b65c-7833848200a9","Type":"ContainerStarted","Data":"ede348d00ccdd3aeab01396358009a022368d6a1fd3cce3e084c4c1f76ed3f1d"}
Jul 27 15:05:54.264054 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:54.263703    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t7xlb" event={"ID":"8af6f3dc-8996-4c73-bfed-716386b74702","Type":"ContainerStarted","Data":"d138c1d63185905ef72e01f865cd492129093608f3f66b6fa41ff99501295a5e"}
Jul 27 15:05:54.264879 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:54.264855    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8hhwl"
Jul 27 15:05:54.265171 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:54.265142    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8hhwl"
Jul 27 15:05:54.291416 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:54.291364    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-46.ec2.internal" podStartSLOduration=23.291342688 podStartE2EDuration="23.291342688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 15:05:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:05:54.27610119 +0000 UTC m=+25.704776376" watchObservedRunningTime="2026-07-27 15:05:54.291342688 +0000 UTC m=+25.720017870"
Jul 27 15:05:54.305363 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:54.305329    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-t7xlb" podStartSLOduration=3.427216559 podStartE2EDuration="24.305319134s" totalImagesPullingTime="20.878102575s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 15:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:05:54.305086395 +0000 UTC m=+25.733761577" watchObservedRunningTime="2026-07-27 15:05:54.305319134 +0000 UTC m=+25.733994317"
Jul 27 15:05:55.057820 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:55.057780    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:55.136017 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:55.135994    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:55.136142 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:55.136102    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:55.136142 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:55.136114    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:55.136258 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:55.136107    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:55.136258 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:55.136207    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:55.136342 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:55.136301    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:55.269299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:55.269232    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp" event={"ID":"6c6a1ca4-48f0-4dee-b65c-7833848200a9","Type":"ContainerStarted","Data":"a6d4d81df56a74739b68c357e52c1be44de912d0b358572d1f216a9f9007a90d"}
Jul 27 15:05:55.273024 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:55.272991    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd" event={"ID":"cfdaf978-f52a-487c-b5b4-d7627b36fed9","Type":"ContainerStarted","Data":"98cff2a11f106b36debbf08c05ff7e364e16597c548ded96eb29b69ea6fd71df"}
Jul 27 15:05:55.300368 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:55.300316    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhvtp" podStartSLOduration=2.172639589 podStartE2EDuration="25.300296103s" totalImagesPullingTime="23.127656514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 15:05:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:05:55.299592806 +0000 UTC m=+26.728267991" watchObservedRunningTime="2026-07-27 15:05:55.300296103 +0000 UTC m=+26.728971266"
Jul 27 15:05:55.456402 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:55.456366    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:55.675127 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:55.675035    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:55.860104 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:55.860071    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:56.061236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:56.061197    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:56.402272 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:56.402204    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:56.786964 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:56.786935    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:57.098148 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:57.098121    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:57.133356 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:57.133331    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:57.133553 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:57.133367    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:57.133553 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:57.133331    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:57.133553 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:57.133464    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:57.133694 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:57.133593    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:57.133694 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:57.133685    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:57.255888 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:57.255848    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:57.464583 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:57.464473    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:05:58.286451 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:58.286232    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd" event={"ID":"cfdaf978-f52a-487c-b5b4-d7627b36fed9","Type":"ContainerStarted","Data":"c187841b514b042e97922e88ffed0f50d77afd2e3311fd066bc062aa6e64da32"}
Jul 27 15:05:58.291992 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:58.291838    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:58.291992 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:58.291906    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:58.310487 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:58.310460    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:58.326554 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:58.325801    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd" podStartSLOduration=7.357038997 podStartE2EDuration="28.325786385s" totalImagesPullingTime="20.968747388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 15:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:05:58.323912978 +0000 UTC m=+29.752588174" watchObservedRunningTime="2026-07-27 15:05:58.325786385 +0000 UTC m=+29.754461568"
Jul 27 15:05:59.133218 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:59.133182    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:05:59.133785 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:59.133261    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:05:59.133785 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:59.133365    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:05:59.133785 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:59.133473    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:05:59.133785 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:59.133555    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:05:59.133785 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:05:59.133651    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:05:59.290268 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:59.290231    2548 generic.go:356] "Generic (PLEG): container finished" podID="92215e75-de40-412c-85a4-0eabf00f5dc1" containerID="43ce7fa1d5ec86b2997fbca5c4a416a01fd831b5de280ae19638d16d8c3bba9f" exitCode=0
Jul 27 15:05:59.290449 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:59.290325    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z4g4" event={"ID":"92215e75-de40-412c-85a4-0eabf00f5dc1","Type":"ContainerDied","Data":"43ce7fa1d5ec86b2997fbca5c4a416a01fd831b5de280ae19638d16d8c3bba9f"}
Jul 27 15:05:59.290954 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:59.290821    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:05:59.307050 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:05:59.307032    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:06:00.071697 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:00.071661    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:00.169845 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:00.169799    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:00.270123 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:00.270092    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:00.860980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:00.860941    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:01.132998 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:01.132960    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:01.133173 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:01.133083    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:01.133173 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:01.133130    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:06:01.133173 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:01.133170    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:01.133307 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:01.133141    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:06:01.133307 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:01.133215    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:06:01.296824 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:01.296790    2548 generic.go:356] "Generic (PLEG): container finished" podID="92215e75-de40-412c-85a4-0eabf00f5dc1" containerID="bd30b14b425baa83351a5f79e3bd4e2096267341c15cff62e8298c59fc4bd32d" exitCode=0
Jul 27 15:06:01.297393 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:01.296878    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z4g4" event={"ID":"92215e75-de40-412c-85a4-0eabf00f5dc1","Type":"ContainerDied","Data":"bd30b14b425baa83351a5f79e3bd4e2096267341c15cff62e8298c59fc4bd32d"}
Jul 27 15:06:01.455087 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:01.455014    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:01.664149 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:01.664106    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:02.766712 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:02.766684    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:02.767011 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:02.766733    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs\") pod \"network-metrics-daemon-x69pt\" (UID: \"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3\") " pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:02.767011 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:02.766825    2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 15:06:02.767011 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:02.766867    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs podName:f21ffbbc-abb4-4ee7-8fbb-8780412fffc3 nodeName:}" failed. No retries permitted until 2026-07-27 15:06:34.766855153 +0000 UTC m=+66.195530313 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs") pod "network-metrics-daemon-x69pt" (UID: "f21ffbbc-abb4-4ee7-8fbb-8780412fffc3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 15:06:02.767011 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:02.766921    2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 15:06:02.767011 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:02.766974    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret podName:f1023b4e-580a-4dc0-8a69-4167ae7cec34 nodeName:}" failed. No retries permitted until 2026-07-27 15:06:34.766961028 +0000 UTC m=+66.195636189 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret") pod "global-pull-secret-syncer-njr4n" (UID: "f1023b4e-580a-4dc0-8a69-4167ae7cec34") : object "kube-system"/"original-pull-secret" not registered
Jul 27 15:06:02.867439 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:02.867360    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hnvx\" (UniqueName: \"kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx\") pod \"network-check-target-r4tc2\" (UID: \"1e7a0804-791d-4fa4-8155-e7830a048c07\") " pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:02.867585 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:02.867467    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 15:06:02.867585 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:02.867484    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 15:06:02.867585 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:02.867492    2548 projected.go:196] Error preparing data for projected volume kube-api-access-7hnvx for pod openshift-network-diagnostics/network-check-target-r4tc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 15:06:02.867585 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:02.867554    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx podName:1e7a0804-791d-4fa4-8155-e7830a048c07 nodeName:}" failed. No retries permitted until 2026-07-27 15:06:34.86752431 +0000 UTC m=+66.296199471 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7hnvx" (UniqueName: "kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx") pod "network-check-target-r4tc2" (UID: "1e7a0804-791d-4fa4-8155-e7830a048c07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 15:06:03.133307 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:03.133223    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:03.133469 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:03.133223    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:03.133469 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:03.133335    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:06:03.133469 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:03.133232    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:03.133469 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:03.133402    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:06:03.133469 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:03.133463    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:06:03.303289 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:03.303255    2548 generic.go:356] "Generic (PLEG): container finished" podID="92215e75-de40-412c-85a4-0eabf00f5dc1" containerID="4a1048d9ae4fe48abe630fb6472f91a03c9ffb24155666ff5e2956b612a54a5b" exitCode=0
Jul 27 15:06:03.303438 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:03.303318    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z4g4" event={"ID":"92215e75-de40-412c-85a4-0eabf00f5dc1","Type":"ContainerDied","Data":"4a1048d9ae4fe48abe630fb6472f91a03c9ffb24155666ff5e2956b612a54a5b"}
Jul 27 15:06:04.354896 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:04.354860    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:04.457079 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:04.457052    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:05.004245 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:05.004188    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:05.132811 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:05.132731    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:05.132811 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:05.132746    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:05.133032 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:05.132873    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:06:05.133032 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:05.132933    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:05.133140 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:05.133039    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:06:05.133140 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:05.133106    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:06:06.362936 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:06.362898    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:06.463494 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:06.463453    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:07.133046 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:07.133011    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:07.133046 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:07.133050    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:07.133293 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:07.133012    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:07.133293 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:07.133124    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:06:07.133293 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:07.133207    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:06:07.133455 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:07.133313    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:06:08.316993 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:08.316952    2548 generic.go:356] "Generic (PLEG): container finished" podID="92215e75-de40-412c-85a4-0eabf00f5dc1" containerID="047f4b4521f585efd5cc711001c9774d0c442754e429b07f1b8cf4372c0953dc" exitCode=0
Jul 27 15:06:08.317331 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:08.317030    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z4g4" event={"ID":"92215e75-de40-412c-85a4-0eabf00f5dc1","Type":"ContainerDied","Data":"047f4b4521f585efd5cc711001c9774d0c442754e429b07f1b8cf4372c0953dc"}
Jul 27 15:06:08.962603 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:08.962405    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:09.070683 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:09.070654    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:09.135480 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:09.135452    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:09.135636 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:09.135554    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:09.135636 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:09.135568    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:06:09.135636 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:09.135579    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:09.135750 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:09.135638    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:06:09.135750 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:09.135690    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:06:09.321611 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:09.321576    2548 generic.go:356] "Generic (PLEG): container finished" podID="92215e75-de40-412c-85a4-0eabf00f5dc1" containerID="463b48505eaf1de5947341a32d2990dea2f43c6aade5086666a620a1476170c2" exitCode=0
Jul 27 15:06:09.321943 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:09.321624    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z4g4" event={"ID":"92215e75-de40-412c-85a4-0eabf00f5dc1","Type":"ContainerDied","Data":"463b48505eaf1de5947341a32d2990dea2f43c6aade5086666a620a1476170c2"}
Jul 27 15:06:10.325393 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:10.325357    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:10.327127 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:10.327102    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z4g4" event={"ID":"92215e75-de40-412c-85a4-0eabf00f5dc1","Type":"ContainerStarted","Data":"de81b27c1556784ce8ffc962c396316674caf397d7dda1d2cf5c80896ae15e0f"}
Jul 27 15:06:10.356540 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:10.356499    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7z4g4" podStartSLOduration=7.894423028 podStartE2EDuration="40.356486391s" totalImagesPullingTime="32.462063363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 15:05:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:06:10.356087201 +0000 UTC m=+41.784762380" watchObservedRunningTime="2026-07-27 15:06:10.356486391 +0000 UTC m=+41.785161574"
Jul 27 15:06:10.401932 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:10.401901    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:11.132700 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:11.132670    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:11.132889 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:11.132670    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:11.132889 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:11.132783    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:06:11.132889 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:11.132680    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:11.132889 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:11.132855    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:06:11.133024 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:11.132948    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:06:13.135432 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:13.135404    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:13.135805 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:13.135490    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:13.135805 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:13.135494    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:06:13.135805 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:13.135518    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:13.135805 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:13.135607    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:06:13.135805 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:13.135670    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:06:13.551161 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:13.551130    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:13.652074 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:13.652037    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:13.740790 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:13.740762    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:14.286890 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:14.286859    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:14.899733 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:14.899701    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:15.133075 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:15.133049    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:15.133285 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:15.133049    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:15.133285 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:15.133055    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:15.133285 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:15.133234    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:06:15.133402 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:15.133141    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:06:15.133402 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:15.133307    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:06:15.139077 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:15.139056    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:15.239683 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:15.239658    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:16.118085 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:16.118034    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:16.218125 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:16.218086    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:16.443024 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:16.442941    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x69pt"]
Jul 27 15:06:16.443164 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:16.443120    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:16.443276 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:16.443250    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:06:16.445734 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:16.445708    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-njr4n"]
Jul 27 15:06:16.446087 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:16.446069    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:16.446342 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:16.446317    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:06:16.446746 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:16.446723    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r4tc2"]
Jul 27 15:06:16.446852 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:16.446814    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:16.446908 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:16.446891    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:06:16.504238 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:16.504210    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:16.636761 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:16.636733    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:16.807040 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:16.807012    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:17.117890 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:17.117798    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:17.229574 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:17.229526    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:18.133225 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:18.133197    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:18.133225 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:18.133220    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:18.133426 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:18.133197    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:18.133426 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:18.133299    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4tc2" podUID="1e7a0804-791d-4fa4-8155-e7830a048c07"
Jul 27 15:06:18.133426 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:18.133374    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x69pt" podUID="f21ffbbc-abb4-4ee7-8fbb-8780412fffc3"
Jul 27 15:06:18.133426 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:18.133423    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njr4n" podUID="f1023b4e-580a-4dc0-8a69-4167ae7cec34"
Jul 27 15:06:18.777267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:18.777234    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:18.876063 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:18.875970    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:19.121251 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.121085    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:19.290723 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.290691    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:19.761183 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.761159    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:19.863177 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.863144    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:19.872848 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.872827    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-46.ec2.internal" event="NodeReady"
Jul 27 15:06:19.872956 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.872951    2548 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 27 15:06:19.925962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.925932    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wnvb6"]
Jul 27 15:06:19.939578 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.939519    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dnp8b"]
Jul 27 15:06:19.939713 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.939661    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wnvb6"
Jul 27 15:06:19.942550 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.942514    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2c97b\""
Jul 27 15:06:19.942941 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.942926    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 15:06:19.943081 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.943065    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 15:06:19.943143 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.943129    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 15:06:19.959653 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.959626    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zrg2s"]
Jul 27 15:06:19.959788 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.959767    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:19.963383 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.963143    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 15:06:19.963383 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.963170    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zc6mm\""
Jul 27 15:06:19.963383 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.963225    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 15:06:19.963383 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.963359    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 15:06:19.963700 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.963589    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 15:06:19.985927 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.985803    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dnp8b"]
Jul 27 15:06:19.985927 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.985861    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zrg2s"]
Jul 27 15:06:19.986140 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.985977    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:19.986524 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.986501    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wnvb6"]
Jul 27 15:06:19.989597 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.989573    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 15:06:19.989786 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.989765    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 15:06:19.989879 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:19.989817    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-spr6f\""
Jul 27 15:06:20.003173 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.003150    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwp7q\" (UniqueName: \"kubernetes.io/projected/3962639b-3a18-4284-9454-f14e7c24f8a3-kube-api-access-wwp7q\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.003313 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.003178    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/752e4bdb-fc3f-4c7f-8c7f-04edea561071-metrics-tls\") pod \"dns-default-zrg2s\" (UID: \"752e4bdb-fc3f-4c7f-8c7f-04edea561071\") " pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:20.003313 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.003197    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3962639b-3a18-4284-9454-f14e7c24f8a3-data-volume\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.003313 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.003219    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46e7ac8e-6e47-48e1-bce2-20f3897c3af8-cert\") pod \"ingress-canary-wnvb6\" (UID: \"46e7ac8e-6e47-48e1-bce2-20f3897c3af8\") " pod="openshift-ingress-canary/ingress-canary-wnvb6"
Jul 27 15:06:20.003313 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.003252    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3962639b-3a18-4284-9454-f14e7c24f8a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.003471 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.003318    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3962639b-3a18-4284-9454-f14e7c24f8a3-crio-socket\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.003471 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.003337    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/752e4bdb-fc3f-4c7f-8c7f-04edea561071-config-volume\") pod \"dns-default-zrg2s\" (UID: \"752e4bdb-fc3f-4c7f-8c7f-04edea561071\") " pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:20.003471 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.003352    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/752e4bdb-fc3f-4c7f-8c7f-04edea561071-tmp-dir\") pod \"dns-default-zrg2s\" (UID: \"752e4bdb-fc3f-4c7f-8c7f-04edea561071\") " pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:20.003471 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.003426    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3962639b-3a18-4284-9454-f14e7c24f8a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.003471 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.003454    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzf7k\" (UniqueName: \"kubernetes.io/projected/46e7ac8e-6e47-48e1-bce2-20f3897c3af8-kube-api-access-zzf7k\") pod \"ingress-canary-wnvb6\" (UID: \"46e7ac8e-6e47-48e1-bce2-20f3897c3af8\") " pod="openshift-ingress-canary/ingress-canary-wnvb6"
Jul 27 15:06:20.003653 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.003476    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9gg5\" (UniqueName: \"kubernetes.io/projected/752e4bdb-fc3f-4c7f-8c7f-04edea561071-kube-api-access-k9gg5\") pod \"dns-default-zrg2s\" (UID: \"752e4bdb-fc3f-4c7f-8c7f-04edea561071\") " pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:20.104153 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.104058    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwp7q\" (UniqueName: \"kubernetes.io/projected/3962639b-3a18-4284-9454-f14e7c24f8a3-kube-api-access-wwp7q\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.104153 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.104090    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/752e4bdb-fc3f-4c7f-8c7f-04edea561071-metrics-tls\") pod \"dns-default-zrg2s\" (UID: \"752e4bdb-fc3f-4c7f-8c7f-04edea561071\") " pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:20.104153 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.104111    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3962639b-3a18-4284-9454-f14e7c24f8a3-data-volume\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.104153 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.104129    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46e7ac8e-6e47-48e1-bce2-20f3897c3af8-cert\") pod \"ingress-canary-wnvb6\" (UID: \"46e7ac8e-6e47-48e1-bce2-20f3897c3af8\") " pod="openshift-ingress-canary/ingress-canary-wnvb6"
Jul 27 15:06:20.104153 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.104145    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3962639b-3a18-4284-9454-f14e7c24f8a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.105236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.104276    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3962639b-3a18-4284-9454-f14e7c24f8a3-crio-socket\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.105236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.104305    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/752e4bdb-fc3f-4c7f-8c7f-04edea561071-config-volume\") pod \"dns-default-zrg2s\" (UID: \"752e4bdb-fc3f-4c7f-8c7f-04edea561071\") " pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:20.105236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.104328    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/752e4bdb-fc3f-4c7f-8c7f-04edea561071-tmp-dir\") pod \"dns-default-zrg2s\" (UID: \"752e4bdb-fc3f-4c7f-8c7f-04edea561071\") " pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:20.105236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.104392    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3962639b-3a18-4284-9454-f14e7c24f8a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.105236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.104424    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf7k\" (UniqueName: \"kubernetes.io/projected/46e7ac8e-6e47-48e1-bce2-20f3897c3af8-kube-api-access-zzf7k\") pod \"ingress-canary-wnvb6\" (UID: \"46e7ac8e-6e47-48e1-bce2-20f3897c3af8\") " pod="openshift-ingress-canary/ingress-canary-wnvb6"
Jul 27 15:06:20.105236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.104456    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9gg5\" (UniqueName: \"kubernetes.io/projected/752e4bdb-fc3f-4c7f-8c7f-04edea561071-kube-api-access-k9gg5\") pod \"dns-default-zrg2s\" (UID: \"752e4bdb-fc3f-4c7f-8c7f-04edea561071\") " pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:20.105236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.104506    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3962639b-3a18-4284-9454-f14e7c24f8a3-data-volume\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.105236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.104564    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3962639b-3a18-4284-9454-f14e7c24f8a3-crio-socket\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.105236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.104776    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/752e4bdb-fc3f-4c7f-8c7f-04edea561071-tmp-dir\") pod \"dns-default-zrg2s\" (UID: \"752e4bdb-fc3f-4c7f-8c7f-04edea561071\") " pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:20.105236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.105006    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/752e4bdb-fc3f-4c7f-8c7f-04edea561071-config-volume\") pod \"dns-default-zrg2s\" (UID: \"752e4bdb-fc3f-4c7f-8c7f-04edea561071\") " pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:20.105236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.105169    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3962639b-3a18-4284-9454-f14e7c24f8a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.107383 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.107358    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:20.108438 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.108420    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/752e4bdb-fc3f-4c7f-8c7f-04edea561071-metrics-tls\") pod \"dns-default-zrg2s\" (UID: \"752e4bdb-fc3f-4c7f-8c7f-04edea561071\") " pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:20.108562 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.108473    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3962639b-3a18-4284-9454-f14e7c24f8a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.108612 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.108567    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46e7ac8e-6e47-48e1-bce2-20f3897c3af8-cert\") pod \"ingress-canary-wnvb6\" (UID: \"46e7ac8e-6e47-48e1-bce2-20f3897c3af8\") " pod="openshift-ingress-canary/ingress-canary-wnvb6"
Jul 27 15:06:20.116997 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.116965    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwp7q\" (UniqueName: \"kubernetes.io/projected/3962639b-3a18-4284-9454-f14e7c24f8a3-kube-api-access-wwp7q\") pod \"insights-runtime-extractor-dnp8b\" (UID: \"3962639b-3a18-4284-9454-f14e7c24f8a3\") " pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.116997 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.116983    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzf7k\" (UniqueName: \"kubernetes.io/projected/46e7ac8e-6e47-48e1-bce2-20f3897c3af8-kube-api-access-zzf7k\") pod \"ingress-canary-wnvb6\" (UID: \"46e7ac8e-6e47-48e1-bce2-20f3897c3af8\") " pod="openshift-ingress-canary/ingress-canary-wnvb6"
Jul 27 15:06:20.119078 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.119051    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9gg5\" (UniqueName: \"kubernetes.io/projected/752e4bdb-fc3f-4c7f-8c7f-04edea561071-kube-api-access-k9gg5\") pod \"dns-default-zrg2s\" (UID: \"752e4bdb-fc3f-4c7f-8c7f-04edea561071\") " pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:20.132858 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.132827    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:20.132858 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.132846    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:20.133030 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.132850    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:20.136033 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.136016    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2bzfc\""
Jul 27 15:06:20.136135 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.136019    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 15:06:20.136277 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.136264    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 15:06:20.137025 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.137008    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 15:06:20.137132 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.137117    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 15:06:20.137254 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.137239    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pkhzc\""
Jul 27 15:06:20.210145 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.210113    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:20.250091 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.250061    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wnvb6"
Jul 27 15:06:20.269272 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.269256    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dnp8b"
Jul 27 15:06:20.294607 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.294582    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:20.467628 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.467597    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dnp8b"]
Jul 27 15:06:20.471158 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.471088    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wnvb6"]
Jul 27 15:06:20.476096 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:20.476029    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zrg2s"]
Jul 27 15:06:20.477831 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:06:20.477222    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e7ac8e_6e47_48e1_bce2_20f3897c3af8.slice/crio-f5b4deaaa94b4d6d8e4b461eedd48e08153ef87d04c223c93437b76c30e7032b WatchSource:0}: Error finding container f5b4deaaa94b4d6d8e4b461eedd48e08153ef87d04c223c93437b76c30e7032b: Status 404 returned error can't find the container with id f5b4deaaa94b4d6d8e4b461eedd48e08153ef87d04c223c93437b76c30e7032b
Jul 27 15:06:20.481236 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:06:20.481214    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod752e4bdb_fc3f_4c7f_8c7f_04edea561071.slice/crio-32f9bcb37415a8e49bbf20ca07f9c1aa603add37e3321ef6e9e2cf3be0834e15 WatchSource:0}: Error finding container 32f9bcb37415a8e49bbf20ca07f9c1aa603add37e3321ef6e9e2cf3be0834e15: Status 404 returned error can't find the container with id 32f9bcb37415a8e49bbf20ca07f9c1aa603add37e3321ef6e9e2cf3be0834e15
Jul 27 15:06:21.355768 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:21.355673    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zrg2s" event={"ID":"752e4bdb-fc3f-4c7f-8c7f-04edea561071","Type":"ContainerStarted","Data":"32f9bcb37415a8e49bbf20ca07f9c1aa603add37e3321ef6e9e2cf3be0834e15"}
Jul 27 15:06:21.357946 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:21.357912    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dnp8b" event={"ID":"3962639b-3a18-4284-9454-f14e7c24f8a3","Type":"ContainerStarted","Data":"31733f2a555d3ad3ff0b19523442a8e8480ab7e73273a1e342610f9585b0e889"}
Jul 27 15:06:21.359723 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:21.359694    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wnvb6" event={"ID":"46e7ac8e-6e47-48e1-bce2-20f3897c3af8","Type":"ContainerStarted","Data":"f5b4deaaa94b4d6d8e4b461eedd48e08153ef87d04c223c93437b76c30e7032b"}
Jul 27 15:06:23.098862 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:23.098684    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:23.190285 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:23.190194    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:23.857455 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:23.857407    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:23.948651 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:23.948611    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:25.053835 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:25.053791    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:25.152828 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:25.152800    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:25.258190 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:25.258099    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:25.357463 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:25.357347    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:25.815456 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:25.815426    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:25.915996 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:25.915953    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:26.375334 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:26.375250    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dnp8b" event={"ID":"3962639b-3a18-4284-9454-f14e7c24f8a3","Type":"ContainerStarted","Data":"a47202948d466cba02fa6fc0f307446ff52f145cf155ec9da847f2fc6b528912"}
Jul 27 15:06:26.377354 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:26.377328    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zrg2s" event={"ID":"752e4bdb-fc3f-4c7f-8c7f-04edea561071","Type":"ContainerStarted","Data":"6de408666f5a61c185011bf6bfda24d26fd4753706cbeacda5713557982bc6d9"}
Jul 27 15:06:26.377517 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:26.377383    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zrg2s" event={"ID":"752e4bdb-fc3f-4c7f-8c7f-04edea561071","Type":"ContainerStarted","Data":"7415ee78a39130602ad0754e00e6a574b1babf5f189b927f5902773ba66dca1e"}
Jul 27 15:06:26.377517 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:26.377470    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:26.395886 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:26.395838    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zrg2s" podStartSLOduration=1.962085261 podStartE2EDuration="7.395822871s" totalImagesPullingTime="5.43373761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:06:26.39486864 +0000 UTC m=+57.823543845" watchObservedRunningTime="2026-07-27 15:06:26.395822871 +0000 UTC m=+57.824498033"
Jul 27 15:06:27.381980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:27.381873    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wnvb6" event={"ID":"46e7ac8e-6e47-48e1-bce2-20f3897c3af8","Type":"ContainerStarted","Data":"9b6286d7cf86454d56d00611da0b2733ff827219a15e57e2af331bf6a07b2f38"}
Jul 27 15:06:27.403641 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:27.403573    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wnvb6" podStartSLOduration=1.864688825 podStartE2EDuration="8.403555974s" totalImagesPullingTime="6.538867149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:06:27.401993921 +0000 UTC m=+58.830669136" watchObservedRunningTime="2026-07-27 15:06:27.403555974 +0000 UTC m=+58.832231158"
Jul 27 15:06:27.948055 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:27.947998    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:28.043303 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.043272    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:28.139909 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.139871    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:28.227845 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.227812    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"]
Jul 27 15:06:28.233377 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.233348    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"
Jul 27 15:06:28.236446 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.236422    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fkrw9\""
Jul 27 15:06:28.236619 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.236482    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 27 15:06:28.236619 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.236486    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 15:06:28.238513 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.238492    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 27 15:06:28.238687 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.238563    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 15:06:28.238687 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.238618    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 15:06:28.240089 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.240060    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:28.248178 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.248150    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"]
Jul 27 15:06:28.263660 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.263610    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gdl67"]
Jul 27 15:06:28.267566 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.267509    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.270616 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.270587    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 15:06:28.270779 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.270591    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 15:06:28.270966 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.270949    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vw8rk\""
Jul 27 15:06:28.271234 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.271214    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cd26152-a496-4c0d-ae7c-040a5b44cb56-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-cc5lw\" (UID: \"6cd26152-a496-4c0d-ae7c-040a5b44cb56\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"
Jul 27 15:06:28.271316 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.271245    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5rx2\" (UniqueName: \"kubernetes.io/projected/6cd26152-a496-4c0d-ae7c-040a5b44cb56-kube-api-access-q5rx2\") pod \"openshift-state-metrics-b8f49f5c6-cc5lw\" (UID: \"6cd26152-a496-4c0d-ae7c-040a5b44cb56\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"
Jul 27 15:06:28.271316 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.271227    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 15:06:28.271316 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.271278    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd26152-a496-4c0d-ae7c-040a5b44cb56-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cc5lw\" (UID: \"6cd26152-a496-4c0d-ae7c-040a5b44cb56\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"
Jul 27 15:06:28.271443 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.271365    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6cd26152-a496-4c0d-ae7c-040a5b44cb56-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-cc5lw\" (UID: \"6cd26152-a496-4c0d-ae7c-040a5b44cb56\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"
Jul 27 15:06:28.344837 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.344798    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:28.371875 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.371834    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6cd26152-a496-4c0d-ae7c-040a5b44cb56-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-cc5lw\" (UID: \"6cd26152-a496-4c0d-ae7c-040a5b44cb56\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"
Jul 27 15:06:28.372042 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.371918    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/120ac523-537f-4733-8449-86d168cba4f7-root\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.372042 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.371949    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.372042 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.371983    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.372042 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.372008    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/120ac523-537f-4733-8449-86d168cba4f7-sys\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.372211 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.372045    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-textfile\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.372211 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.372093    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vlkf\" (UniqueName: \"kubernetes.io/projected/120ac523-537f-4733-8449-86d168cba4f7-kube-api-access-7vlkf\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.372291 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.372214    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cd26152-a496-4c0d-ae7c-040a5b44cb56-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-cc5lw\" (UID: \"6cd26152-a496-4c0d-ae7c-040a5b44cb56\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"
Jul 27 15:06:28.372291 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.372241    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5rx2\" (UniqueName: \"kubernetes.io/projected/6cd26152-a496-4c0d-ae7c-040a5b44cb56-kube-api-access-q5rx2\") pod \"openshift-state-metrics-b8f49f5c6-cc5lw\" (UID: \"6cd26152-a496-4c0d-ae7c-040a5b44cb56\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"
Jul 27 15:06:28.372291 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.372264    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-wtmp\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.372427 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.372288    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/120ac523-537f-4733-8449-86d168cba4f7-metrics-client-ca\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.372427 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.372321    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd26152-a496-4c0d-ae7c-040a5b44cb56-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cc5lw\" (UID: \"6cd26152-a496-4c0d-ae7c-040a5b44cb56\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"
Jul 27 15:06:28.372427 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.372348    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-tls\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.372994 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.372959    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cd26152-a496-4c0d-ae7c-040a5b44cb56-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-cc5lw\" (UID: \"6cd26152-a496-4c0d-ae7c-040a5b44cb56\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"
Jul 27 15:06:28.375528 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.375500    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd26152-a496-4c0d-ae7c-040a5b44cb56-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cc5lw\" (UID: \"6cd26152-a496-4c0d-ae7c-040a5b44cb56\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"
Jul 27 15:06:28.375658 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.375596    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6cd26152-a496-4c0d-ae7c-040a5b44cb56-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-cc5lw\" (UID: \"6cd26152-a496-4c0d-ae7c-040a5b44cb56\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"
Jul 27 15:06:28.382920 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.382888    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5rx2\" (UniqueName: \"kubernetes.io/projected/6cd26152-a496-4c0d-ae7c-040a5b44cb56-kube-api-access-q5rx2\") pod \"openshift-state-metrics-b8f49f5c6-cc5lw\" (UID: \"6cd26152-a496-4c0d-ae7c-040a5b44cb56\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"
Jul 27 15:06:28.445355 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.445325    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:28.473453 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.473421    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.473645 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.473460    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/120ac523-537f-4733-8449-86d168cba4f7-sys\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.473645 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.473500    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-textfile\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.473645 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.473524    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vlkf\" (UniqueName: \"kubernetes.io/projected/120ac523-537f-4733-8449-86d168cba4f7-kube-api-access-7vlkf\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.473645 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.473578    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-wtmp\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.473645 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.473629    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/120ac523-537f-4733-8449-86d168cba4f7-sys\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.473895 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.473702    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/120ac523-537f-4733-8449-86d168cba4f7-metrics-client-ca\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.473895 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.473739    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-tls\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.473895 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.473823    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/120ac523-537f-4733-8449-86d168cba4f7-root\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.473895 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.473861    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/120ac523-537f-4733-8449-86d168cba4f7-root\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.473895 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.473870    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.474241 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.473703    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-wtmp\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.474241 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.473915    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-textfile\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.474636 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.474612    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/120ac523-537f-4733-8449-86d168cba4f7-metrics-client-ca\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.474736 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.474702    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.476347 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.476325    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.476781 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.476759    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/120ac523-537f-4733-8449-86d168cba4f7-node-exporter-tls\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.484291 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.484218    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vlkf\" (UniqueName: \"kubernetes.io/projected/120ac523-537f-4733-8449-86d168cba4f7-kube-api-access-7vlkf\") pod \"node-exporter-gdl67\" (UID: \"120ac523-537f-4733-8449-86d168cba4f7\") " pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.545141 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.545113    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"
Jul 27 15:06:28.545918 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.545894    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:28.579177 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.579136    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gdl67"
Jul 27 15:06:28.645559 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.645513    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:28.746587 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:28.746483    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:29.182355 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.182275    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:29.279208 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.279171    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:29.286838 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.286817    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 15:06:29.298306 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.298282    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.301222 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.301201    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 27 15:06:29.301444 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.301425    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jg9r5\""
Jul 27 15:06:29.309740 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.309722    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 27 15:06:29.310504 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.310489    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 27 15:06:29.310850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.310794    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 27 15:06:29.310850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.310826    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 27 15:06:29.311003 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.310848    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 27 15:06:29.311003 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.310802    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 27 15:06:29.311003 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.310970    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 27 15:06:29.311003 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.310822    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 27 15:06:29.316514 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.316491    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 15:06:29.380517 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.380479    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/52583d13-4407-4992-930e-84194a7aef1b-config-out\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.380683 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.380521    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-web-config\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.380683 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.380639    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-config-volume\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.380767 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.380678    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6mpb\" (UniqueName: \"kubernetes.io/projected/52583d13-4407-4992-930e-84194a7aef1b-kube-api-access-j6mpb\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.380767 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.380716    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.380869 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.380770    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.380869 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.380825    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.380869 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.380859    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.381007 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.380891    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/52583d13-4407-4992-930e-84194a7aef1b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.381007 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.380915    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.381007 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.380959    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.381007 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.380979    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.381166 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.381010    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.481611 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.481577    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.481611 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.481617    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.482132 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.481643    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.482132 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.481776    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.482132 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.481819    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/52583d13-4407-4992-930e-84194a7aef1b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.482132 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.481849    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.482132 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.481893    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.482132 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.481925    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.482132 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.481957    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.482132 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.482021    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/52583d13-4407-4992-930e-84194a7aef1b-config-out\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.482132 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.482052    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-web-config\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.482132 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.482083    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-config-volume\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.482132 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.482108    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6mpb\" (UniqueName: \"kubernetes.io/projected/52583d13-4407-4992-930e-84194a7aef1b-kube-api-access-j6mpb\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.482700 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:29.482310    2548 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 27 15:06:29.482700 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:29.482373    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-main-tls podName:52583d13-4407-4992-930e-84194a7aef1b nodeName:}" failed. No retries permitted until 2026-07-27 15:06:29.982353184 +0000 UTC m=+61.411028360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "52583d13-4407-4992-930e-84194a7aef1b") : secret "alertmanager-main-tls" not found
Jul 27 15:06:29.482700 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.482369    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.482700 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.482428    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.482700 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:29.482576    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-trusted-ca-bundle podName:52583d13-4407-4992-930e-84194a7aef1b nodeName:}" failed. No retries permitted until 2026-07-27 15:06:29.982555896 +0000 UTC m=+61.411231073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "52583d13-4407-4992-930e-84194a7aef1b") : configmap references non-existent config key: ca-bundle.crt
Jul 27 15:06:29.485621 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.485581    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.485621 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.485582    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/52583d13-4407-4992-930e-84194a7aef1b-config-out\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.486088 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.485822    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/52583d13-4407-4992-930e-84194a7aef1b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.486088 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.485891    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-web-config\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.486232 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.486213    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.486301 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.486280    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.486627 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.486608    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-config-volume\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.494011 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.493993    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6mpb\" (UniqueName: \"kubernetes.io/projected/52583d13-4407-4992-930e-84194a7aef1b-kube-api-access-j6mpb\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.498305 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.498180    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.745093 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.745012    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:29.989179 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.989139    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.989418 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.989230    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.989980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.989956    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:29.991570 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:29.991551    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:30.002383 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:06:30.002358    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod120ac523_537f_4733_8449_86d168cba4f7.slice/crio-bf3bd2975ee2c8ad78d3ff98e2fd4ac98e04924662bd22d3391d64ba54693a95 WatchSource:0}: Error finding container bf3bd2975ee2c8ad78d3ff98e2fd4ac98e04924662bd22d3391d64ba54693a95: Status 404 returned error can't find the container with id bf3bd2975ee2c8ad78d3ff98e2fd4ac98e04924662bd22d3391d64ba54693a95
Jul 27 15:06:30.118323 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:30.118297    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw"]
Jul 27 15:06:30.121009 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:06:30.120969    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd26152_a496_4c0d_ae7c_040a5b44cb56.slice/crio-338445c2ebaeb8008f4555797a42d6b62b2a94bcec74d8226f33cf7da91f8c00 WatchSource:0}: Error finding container 338445c2ebaeb8008f4555797a42d6b62b2a94bcec74d8226f33cf7da91f8c00: Status 404 returned error can't find the container with id 338445c2ebaeb8008f4555797a42d6b62b2a94bcec74d8226f33cf7da91f8c00
Jul 27 15:06:30.209827 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:30.209806    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:06:30.372419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:30.372383    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 15:06:30.376474 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:06:30.376450    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52583d13_4407_4992_930e_84194a7aef1b.slice/crio-593e0e33256f539290eed8532faa965a4591f7ab22ec1e1bf264491699d131fb WatchSource:0}: Error finding container 593e0e33256f539290eed8532faa965a4591f7ab22ec1e1bf264491699d131fb: Status 404 returned error can't find the container with id 593e0e33256f539290eed8532faa965a4591f7ab22ec1e1bf264491699d131fb
Jul 27 15:06:30.391979 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:30.391950    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerStarted","Data":"593e0e33256f539290eed8532faa965a4591f7ab22ec1e1bf264491699d131fb"}
Jul 27 15:06:30.393669 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:30.393646    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dnp8b" event={"ID":"3962639b-3a18-4284-9454-f14e7c24f8a3","Type":"ContainerStarted","Data":"b58c1211e4cd909d2cb6c4a86c70f2220b9b9454fa07f5d2b4583a095b7c6055"}
Jul 27 15:06:30.395355 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:30.395330    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw" event={"ID":"6cd26152-a496-4c0d-ae7c-040a5b44cb56","Type":"ContainerStarted","Data":"cf1f04198eb43d38a767e3ccc6c970e0f52dc1f683707b5ae3fe91aeab9f8c85"}
Jul 27 15:06:30.395451 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:30.395375    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw" event={"ID":"6cd26152-a496-4c0d-ae7c-040a5b44cb56","Type":"ContainerStarted","Data":"ef20380270b218aa5f3bfa66cc7fedc32065627b4fb8e80a5a4a0691cf4aa05c"}
Jul 27 15:06:30.395451 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:30.395401    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw" event={"ID":"6cd26152-a496-4c0d-ae7c-040a5b44cb56","Type":"ContainerStarted","Data":"338445c2ebaeb8008f4555797a42d6b62b2a94bcec74d8226f33cf7da91f8c00"}
Jul 27 15:06:30.396382 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:30.396364    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gdl67" event={"ID":"120ac523-537f-4733-8449-86d168cba4f7","Type":"ContainerStarted","Data":"bf3bd2975ee2c8ad78d3ff98e2fd4ac98e04924662bd22d3391d64ba54693a95"}
Jul 27 15:06:31.187068 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:31.187034    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:31.274066 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:31.274036    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:31.317758 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:31.317732    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j94kd"
Jul 27 15:06:31.364217 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:31.364184    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:31.435162 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:31.435087    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:31.528480 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:31.528414    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:31.629386 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:31.629335    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:32.406907 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.406869    2548 generic.go:356] "Generic (PLEG): container finished" podID="52583d13-4407-4992-930e-84194a7aef1b" containerID="10dd7ac226b368be5b0f70d4eb56823a5a30e1e9defc9c72e383ee0c60985130" exitCode=0
Jul 27 15:06:32.407406 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.406923    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerDied","Data":"10dd7ac226b368be5b0f70d4eb56823a5a30e1e9defc9c72e383ee0c60985130"}
Jul 27 15:06:32.409481 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.409452    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw" event={"ID":"6cd26152-a496-4c0d-ae7c-040a5b44cb56","Type":"ContainerStarted","Data":"c7425b83bcb48c5228cd0b3fc2c5272b3898e4094f91adb2e714be485d823d98"}
Jul 27 15:06:32.463437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.463390    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cc5lw" podStartSLOduration=2.910945709 podStartE2EDuration="4.463372957s" totalImagesPullingTime="1.552427248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:06:32.462948557 +0000 UTC m=+63.891623744" watchObservedRunningTime="2026-07-27 15:06:32.463372957 +0000 UTC m=+63.892048139"
Jul 27 15:06:32.620375 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.620336    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-9455d7f8-4lfdc"]
Jul 27 15:06:32.624076 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.624052    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.627055 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.627028    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-gxxwk\""
Jul 27 15:06:32.627220 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.627127    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 27 15:06:32.627220 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.627161    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 27 15:06:32.627491 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.627463    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 27 15:06:32.627696 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.627679    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 27 15:06:32.627786 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.627753    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-chs5l7qipgj66\""
Jul 27 15:06:32.636313 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.636290    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-9455d7f8-4lfdc"]
Jul 27 15:06:32.712254 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.712164    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c851e1cb-42ea-453f-a7e8-2735be4fc62a-secret-metrics-server-tls\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.712254 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.712201    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5rg2\" (UniqueName: \"kubernetes.io/projected/c851e1cb-42ea-453f-a7e8-2735be4fc62a-kube-api-access-p5rg2\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.712254 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.712225    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c851e1cb-42ea-453f-a7e8-2735be4fc62a-secret-metrics-server-client-certs\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.712563 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.712374    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c851e1cb-42ea-453f-a7e8-2735be4fc62a-metrics-server-audit-profiles\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.712563 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.712400    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c851e1cb-42ea-453f-a7e8-2735be4fc62a-client-ca-bundle\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.712563 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.712424    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c851e1cb-42ea-453f-a7e8-2735be4fc62a-audit-log\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.712563 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.712466    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c851e1cb-42ea-453f-a7e8-2735be4fc62a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.812887 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.812850    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c851e1cb-42ea-453f-a7e8-2735be4fc62a-metrics-server-audit-profiles\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.812887 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.812891    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c851e1cb-42ea-453f-a7e8-2735be4fc62a-client-ca-bundle\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.813138 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.812919    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c851e1cb-42ea-453f-a7e8-2735be4fc62a-audit-log\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.813138 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.812945    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c851e1cb-42ea-453f-a7e8-2735be4fc62a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.813138 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.813016    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c851e1cb-42ea-453f-a7e8-2735be4fc62a-secret-metrics-server-tls\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.813138 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.813047    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5rg2\" (UniqueName: \"kubernetes.io/projected/c851e1cb-42ea-453f-a7e8-2735be4fc62a-kube-api-access-p5rg2\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.813138 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.813079    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c851e1cb-42ea-453f-a7e8-2735be4fc62a-secret-metrics-server-client-certs\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.813599 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.813512    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c851e1cb-42ea-453f-a7e8-2735be4fc62a-audit-log\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.813753 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.813729    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c851e1cb-42ea-453f-a7e8-2735be4fc62a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.814140 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.814115    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c851e1cb-42ea-453f-a7e8-2735be4fc62a-metrics-server-audit-profiles\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.815932 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.815908    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c851e1cb-42ea-453f-a7e8-2735be4fc62a-client-ca-bundle\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.816040 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.815966    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c851e1cb-42ea-453f-a7e8-2735be4fc62a-secret-metrics-server-tls\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.816135 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.816111    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c851e1cb-42ea-453f-a7e8-2735be4fc62a-secret-metrics-server-client-certs\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.821793 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.821772    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5rg2\" (UniqueName: \"kubernetes.io/projected/c851e1cb-42ea-453f-a7e8-2735be4fc62a-kube-api-access-p5rg2\") pod \"metrics-server-9455d7f8-4lfdc\" (UID: \"c851e1cb-42ea-453f-a7e8-2735be4fc62a\") " pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.880718 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.880687    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:32.936522 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.936480    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:32.988797 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:32.988764    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:33.012783 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.012756    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"]
Jul 27 15:06:33.015571 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.015546    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"
Jul 27 15:06:33.018030 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.018008    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hn7wd\""
Jul 27 15:06:33.018359 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.018334    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 27 15:06:33.058433 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.058409    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c594d976d-4vgn5"]
Jul 27 15:06:33.060396 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.060377    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.062603 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.062570    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"]
Jul 27 15:06:33.063110 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.063088    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 27 15:06:33.063306 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.063284    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 27 15:06:33.064578 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.064558    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 27 15:06:33.064685 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.064597    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 27 15:06:33.064745 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.064735    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 27 15:06:33.065334 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.065314    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-b9qxj\""
Jul 27 15:06:33.066030 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.066009    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 27 15:06:33.066180 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.066161    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 27 15:06:33.070075 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.070052    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 27 15:06:33.075574 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.075554    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c594d976d-4vgn5"]
Jul 27 15:06:33.080748 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.080719    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:33.116360 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.116332    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-7xbtq\" (UID: \"8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"
Jul 27 15:06:33.116360 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.116374    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c882f45-d54e-47ef-956f-9c45950db823-console-serving-cert\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.116612 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.116403    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-service-ca\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.116612 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.116517    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c882f45-d54e-47ef-956f-9c45950db823-console-oauth-config\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.116612 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.116599    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbmqq\" (UniqueName: \"kubernetes.io/projected/8c882f45-d54e-47ef-956f-9c45950db823-kube-api-access-vbmqq\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.116770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.116665    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-trusted-ca-bundle\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.116770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.116713    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-console-config\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.116770 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.116759    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-oauth-serving-cert\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.116903 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.116849    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8ddv\" (UniqueName: \"kubernetes.io/projected/8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7-kube-api-access-f8ddv\") pod \"monitoring-plugin-68b7fc9574-7xbtq\" (UID: \"8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"
Jul 27 15:06:33.217895 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.217855    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c882f45-d54e-47ef-956f-9c45950db823-console-oauth-config\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.218080 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.217912    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbmqq\" (UniqueName: \"kubernetes.io/projected/8c882f45-d54e-47ef-956f-9c45950db823-kube-api-access-vbmqq\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.218080 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.217954    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-trusted-ca-bundle\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.218080 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.217976    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-console-config\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.218080 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.218011    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-oauth-serving-cert\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.218289 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.218138    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8ddv\" (UniqueName: \"kubernetes.io/projected/8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7-kube-api-access-f8ddv\") pod \"monitoring-plugin-68b7fc9574-7xbtq\" (UID: \"8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"
Jul 27 15:06:33.218289 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.218195    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-7xbtq\" (UID: \"8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"
Jul 27 15:06:33.218289 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.218229    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c882f45-d54e-47ef-956f-9c45950db823-console-serving-cert\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.218289 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.218257    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-service-ca\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.218476 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:33.218399    2548 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 27 15:06:33.218476 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:33.218461    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7-monitoring-plugin-cert podName:8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7 nodeName:}" failed. No retries permitted until 2026-07-27 15:06:33.71844242 +0000 UTC m=+65.147117583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-7xbtq" (UID: "8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7") : secret "monitoring-plugin-cert" not found
Jul 27 15:06:33.218837 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.218798    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-oauth-serving-cert\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.218977 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.218956    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-trusted-ca-bundle\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.219049 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.218981    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-service-ca\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.219316 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.219291    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-console-config\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.221302 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.221283    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c882f45-d54e-47ef-956f-9c45950db823-console-oauth-config\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.221388 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.221322    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c882f45-d54e-47ef-956f-9c45950db823-console-serving-cert\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.231479 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.231429    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbmqq\" (UniqueName: \"kubernetes.io/projected/8c882f45-d54e-47ef-956f-9c45950db823-kube-api-access-vbmqq\") pod \"console-6c594d976d-4vgn5\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") " pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.232826 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.232798    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8ddv\" (UniqueName: \"kubernetes.io/projected/8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7-kube-api-access-f8ddv\") pod \"monitoring-plugin-68b7fc9574-7xbtq\" (UID: \"8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"
Jul 27 15:06:33.371431 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.371342    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:33.721951 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.721863    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-7xbtq\" (UID: \"8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"
Jul 27 15:06:33.722383 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:33.722002    2548 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 27 15:06:33.722383 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:06:33.722083    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7-monitoring-plugin-cert podName:8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7 nodeName:}" failed. No retries permitted until 2026-07-27 15:06:34.722060126 +0000 UTC m=+66.150735302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-7xbtq" (UID: "8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7") : secret "monitoring-plugin-cert" not found
Jul 27 15:06:33.845250 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.845212    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:33.946593 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:33.946519    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:34.016784 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.016756    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c594d976d-4vgn5"]
Jul 27 15:06:34.020635 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:06:34.020601    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c882f45_d54e_47ef_956f_9c45950db823.slice/crio-1e6300c0ab2c9a02b40aa9f4e112cd9426accf6489218f96bdae6529f5a545fa WatchSource:0}: Error finding container 1e6300c0ab2c9a02b40aa9f4e112cd9426accf6489218f96bdae6529f5a545fa: Status 404 returned error can't find the container with id 1e6300c0ab2c9a02b40aa9f4e112cd9426accf6489218f96bdae6529f5a545fa
Jul 27 15:06:34.035045 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.035019    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-9455d7f8-4lfdc"]
Jul 27 15:06:34.038401 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:06:34.038364    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc851e1cb_42ea_453f_a7e8_2735be4fc62a.slice/crio-1d6042353b2a877ceb9ebc1c0f8a31ccca5e21774718146a7cba2e3468a62d76 WatchSource:0}: Error finding container 1d6042353b2a877ceb9ebc1c0f8a31ccca5e21774718146a7cba2e3468a62d76: Status 404 returned error can't find the container with id 1d6042353b2a877ceb9ebc1c0f8a31ccca5e21774718146a7cba2e3468a62d76
Jul 27 15:06:34.111703 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.111660    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:34.208151 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.208116    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:34.419044 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.419007    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c594d976d-4vgn5" event={"ID":"8c882f45-d54e-47ef-956f-9c45950db823","Type":"ContainerStarted","Data":"1e6300c0ab2c9a02b40aa9f4e112cd9426accf6489218f96bdae6529f5a545fa"}
Jul 27 15:06:34.421548 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.421494    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc" event={"ID":"c851e1cb-42ea-453f-a7e8-2735be4fc62a","Type":"ContainerStarted","Data":"1d6042353b2a877ceb9ebc1c0f8a31ccca5e21774718146a7cba2e3468a62d76"}
Jul 27 15:06:34.424928 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.424902    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dnp8b" event={"ID":"3962639b-3a18-4284-9454-f14e7c24f8a3","Type":"ContainerStarted","Data":"812eaff370c1daa4fca0f6e22d871a1b9f214bfed0b3104028ab22a0fcd61506"}
Jul 27 15:06:34.453168 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.453113    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dnp8b" podStartSLOduration=2.239118562 podStartE2EDuration="15.453098863s" totalImagesPullingTime="13.213980301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 15:06:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:06:34.451409106 +0000 UTC m=+65.880084291" watchObservedRunningTime="2026-07-27 15:06:34.453098863 +0000 UTC m=+65.881774077"
Jul 27 15:06:34.514402 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.514363    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 15:06:34.519607 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.518456    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.525299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.521237    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zpbrk\""
Jul 27 15:06:34.525299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.521493    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 27 15:06:34.525299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.521720    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 27 15:06:34.525299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.521995    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 27 15:06:34.525299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.522298    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 27 15:06:34.525299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.522480    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 27 15:06:34.525299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.522710    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 27 15:06:34.525299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.523484    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 27 15:06:34.525299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.523728    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 27 15:06:34.525299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.523902    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 27 15:06:34.525299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.524075    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8npb9fcu135uc\""
Jul 27 15:06:34.526107 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.526011    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 27 15:06:34.526236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.526208    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 27 15:06:34.526293 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.526249    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 27 15:06:34.526293 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.526285    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 27 15:06:34.526386 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.526249    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 27 15:06:34.527165 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.527140    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:34.540035 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.539949    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.628796    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.628847    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.628884    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2ced953-3363-44c8-8bcb-2684fa915e40-config-out\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.628908    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629007    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629057    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629122    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2ced953-3363-44c8-8bcb-2684fa915e40-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629148    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629175    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-config\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629201    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629233    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629278    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629329    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629354    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629402    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.629817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629446    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.630806 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629469    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mck8k\" (UniqueName: \"kubernetes.io/projected/d2ced953-3363-44c8-8bcb-2684fa915e40-kube-api-access-mck8k\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.630806 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629547    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.630806 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629571    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-web-config\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.630806 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.629593    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.730789    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2ced953-3363-44c8-8bcb-2684fa915e40-config-out\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.730835    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.730867    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.730894    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.730951    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2ced953-3363-44c8-8bcb-2684fa915e40-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.730975    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731001    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-config\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731027    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731104    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731149    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731199    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731224    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731264    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731296    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731366    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mck8k\" (UniqueName: \"kubernetes.io/projected/d2ced953-3363-44c8-8bcb-2684fa915e40-kube-api-access-mck8k\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731399    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-7xbtq\" (UID: \"8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"
Jul 27 15:06:34.732294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731436    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.733559 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731459    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-web-config\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.733559 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731483    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.733559 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731523    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.733559 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731575    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.733559 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.731991    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.733559 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.732793    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.733905 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.733637    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.735494 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.735467    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.737257 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.737229    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.740382 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.740334    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.743382 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.743265    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.743550 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.743434    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.744383 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.744362    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.744773 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.744746    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.745593 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.745575    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.745970 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.745767    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.747932 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.747905    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.748025 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.747998    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.748643 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.748601    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.749250 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.749191    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-web-config\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.749680 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.749658    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2ced953-3363-44c8-8bcb-2684fa915e40-config-out\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.750109 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.750071    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2ced953-3363-44c8-8bcb-2684fa915e40-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.750697 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.750663    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-7xbtq\" (UID: \"8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"
Jul 27 15:06:34.751649 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.751608    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-config\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.753952 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.753934    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mck8k\" (UniqueName: \"kubernetes.io/projected/d2ced953-3363-44c8-8bcb-2684fa915e40-kube-api-access-mck8k\") pod \"prometheus-k8s-0\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.825978 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.825935    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"
Jul 27 15:06:34.833260 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.832568    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:34.833260 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.832641    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs\") pod \"network-metrics-daemon-x69pt\" (UID: \"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3\") " pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:34.835896 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.835664    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 15:06:34.835896 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.835680    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 15:06:34.836908 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.836884    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:34.845804 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.845783    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1023b4e-580a-4dc0-8a69-4167ae7cec34-original-pull-secret\") pod \"global-pull-secret-syncer-njr4n\" (UID: \"f1023b4e-580a-4dc0-8a69-4167ae7cec34\") " pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:34.846851 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.846829    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f21ffbbc-abb4-4ee7-8fbb-8780412fffc3-metrics-certs\") pod \"network-metrics-daemon-x69pt\" (UID: \"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3\") " pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:34.853288 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.853195    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njr4n"
Jul 27 15:06:34.934067 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.934028    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hnvx\" (UniqueName: \"kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx\") pod \"network-check-target-r4tc2\" (UID: \"1e7a0804-791d-4fa4-8155-e7830a048c07\") " pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:34.938110 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.937900    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 15:06:34.947467 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.947441    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 15:06:34.958567 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:34.958492    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hnvx\" (UniqueName: \"kubernetes.io/projected/1e7a0804-791d-4fa4-8155-e7830a048c07-kube-api-access-7hnvx\") pod \"network-check-target-r4tc2\" (UID: \"1e7a0804-791d-4fa4-8155-e7830a048c07\") " pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:35.145802 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:35.145565    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pkhzc\""
Jul 27 15:06:35.159154 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:35.153993    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2bzfc\""
Jul 27 15:06:35.159154 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:35.154245    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x69pt"
Jul 27 15:06:35.159385 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:35.159176    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:35.522616 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:35.522487    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"]
Jul 27 15:06:35.618454 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:35.618414    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r4tc2"]
Jul 27 15:06:35.847262 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:35.847187    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-njr4n"]
Jul 27 15:06:35.849626 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:35.849600    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x69pt"]
Jul 27 15:06:35.861031 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:35.860980    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 15:06:36.160088 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:36.160055    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:36.255832 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:06:36.255800    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1023b4e_580a_4dc0_8a69_4167ae7cec34.slice/crio-5008886314db5f49788f92e1fecb93b5f1c515a15ae4831aa255b0dade8f8f05 WatchSource:0}: Error finding container 5008886314db5f49788f92e1fecb93b5f1c515a15ae4831aa255b0dade8f8f05: Status 404 returned error can't find the container with id 5008886314db5f49788f92e1fecb93b5f1c515a15ae4831aa255b0dade8f8f05
Jul 27 15:06:36.256585 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:06:36.256562    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2ced953_3363_44c8_8bcb_2684fa915e40.slice/crio-0692eba3d1d77378f79d45d4a29bee27dedd4b60b4655467679b413d8beab1f0 WatchSource:0}: Error finding container 0692eba3d1d77378f79d45d4a29bee27dedd4b60b4655467679b413d8beab1f0: Status 404 returned error can't find the container with id 0692eba3d1d77378f79d45d4a29bee27dedd4b60b4655467679b413d8beab1f0
Jul 27 15:06:36.257173 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:36.257148    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:36.384047 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:36.384014    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zrg2s"
Jul 27 15:06:36.439690 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:36.439584    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq" event={"ID":"8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7","Type":"ContainerStarted","Data":"c6eda81c6793cc4421ef852c825e827fdb9ebb7d84cbc0c3f6d5bb8302bb75ea"}
Jul 27 15:06:36.441678 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:36.441597    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerStarted","Data":"0692eba3d1d77378f79d45d4a29bee27dedd4b60b4655467679b413d8beab1f0"}
Jul 27 15:06:36.443828 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:36.443803    2548 generic.go:356] "Generic (PLEG): container finished" podID="120ac523-537f-4733-8449-86d168cba4f7" containerID="d43bf77f83ef263dca7c6a979f135a7e4f524f4078823f385b257b05c77a0076" exitCode=0
Jul 27 15:06:36.443941 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:36.443875    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gdl67" event={"ID":"120ac523-537f-4733-8449-86d168cba4f7","Type":"ContainerDied","Data":"d43bf77f83ef263dca7c6a979f135a7e4f524f4078823f385b257b05c77a0076"}
Jul 27 15:06:36.446063 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:36.446037    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-njr4n" event={"ID":"f1023b4e-580a-4dc0-8a69-4167ae7cec34","Type":"ContainerStarted","Data":"5008886314db5f49788f92e1fecb93b5f1c515a15ae4831aa255b0dade8f8f05"}
Jul 27 15:06:36.447510 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:36.447485    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r4tc2" event={"ID":"1e7a0804-791d-4fa4-8155-e7830a048c07","Type":"ContainerStarted","Data":"22f6a7201d4d11ee27f025512a1c9eef4e0a8aaef866dc143627e65dc6dc4f7b"}
Jul 27 15:06:36.452932 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:36.452906    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerStarted","Data":"f12948252e7849dc203c55a2254b6ae65bf957fe3b9ba41f1feb64e182c109be"}
Jul 27 15:06:36.453053 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:36.452980    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerStarted","Data":"a9ff653e0b4a6b401e4affff90a8b08ea301cf197c12f489f00ceefa18e0af96"}
Jul 27 15:06:36.453053 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:36.453032    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerStarted","Data":"0fd09f6dd661bc79d108df1d93d61b637e37bacf452522260a1eadcbfe142038"}
Jul 27 15:06:36.453151 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:36.453069    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerStarted","Data":"ce8b2f1276901b2d4a11fc73e862742dfe80161e60e7f6daf133c60f78663ab3"}
Jul 27 15:06:37.646965 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:06:37.646929    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf21ffbbc_abb4_4ee7_8fbb_8780412fffc3.slice/crio-f4d6f3d0474ade380bd31440573a4661f591d707f25c00dcb80d0a63bf8cd5b1 WatchSource:0}: Error finding container f4d6f3d0474ade380bd31440573a4661f591d707f25c00dcb80d0a63bf8cd5b1: Status 404 returned error can't find the container with id f4d6f3d0474ade380bd31440573a4661f591d707f25c00dcb80d0a63bf8cd5b1
Jul 27 15:06:38.465855 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:38.465771    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x69pt" event={"ID":"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3","Type":"ContainerStarted","Data":"f4d6f3d0474ade380bd31440573a4661f591d707f25c00dcb80d0a63bf8cd5b1"}
Jul 27 15:06:39.553427 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:39.553390    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:39.654740 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:39.654351    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:40.476128 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.476101    2548 generic.go:356] "Generic (PLEG): container finished" podID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerID="367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383" exitCode=0
Jul 27 15:06:40.476254 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.476189    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerDied","Data":"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"}
Jul 27 15:06:40.479454 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.479425    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gdl67" event={"ID":"120ac523-537f-4733-8449-86d168cba4f7","Type":"ContainerStarted","Data":"2bd8733d8106c83a81529252ce09eb09c688e6dab20275d57b9be5208c5e26e7"}
Jul 27 15:06:40.479577 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.479486    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gdl67" event={"ID":"120ac523-537f-4733-8449-86d168cba4f7","Type":"ContainerStarted","Data":"4115c67dfbb8c033f5a9c098742ca512cc286f9b8d9d3142116d0dd6229976cc"}
Jul 27 15:06:40.481826 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.481700    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c594d976d-4vgn5" event={"ID":"8c882f45-d54e-47ef-956f-9c45950db823","Type":"ContainerStarted","Data":"7ae1c6b3e303750be8befa0d7131a61d6ad53118467e3613341f35674c76dd5c"}
Jul 27 15:06:40.484406 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.483847    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r4tc2" event={"ID":"1e7a0804-791d-4fa4-8155-e7830a048c07","Type":"ContainerStarted","Data":"c5c9ab1853242bf9c8bbe775010fb02178217ada0cd2cf0bbe8e3d44c27185b5"}
Jul 27 15:06:40.484406 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.484220    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:06:40.490034 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.490013    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerStarted","Data":"b8c0cd27101b891c6f7d353c6672a3dfd43621b969128942f623725cd6a8b410"}
Jul 27 15:06:40.492162 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.491781    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc" event={"ID":"c851e1cb-42ea-453f-a7e8-2735be4fc62a","Type":"ContainerStarted","Data":"195741677c0da20f0b42e32a227800851337bbc3bb75d7e69213eaab3508b95f"}
Jul 27 15:06:40.494096 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.494075    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq" event={"ID":"8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7","Type":"ContainerStarted","Data":"5965bad19dfc07eb93bb24110abd4182251ce39ffaa61e6d40afb9d9268deafc"}
Jul 27 15:06:40.494578 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.494457    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"
Jul 27 15:06:40.500495 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.500455    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq"
Jul 27 15:06:40.545037 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.544983    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gdl67" podStartSLOduration=7.079101203 podStartE2EDuration="12.544968714s" totalImagesPullingTime="5.339294307s" totalInitContainerRuntime="126.573204ms" isStatefulPod=true podCreationTimestamp="2026-07-27 15:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:06:40.544381157 +0000 UTC m=+71.973056333" watchObservedRunningTime="2026-07-27 15:06:40.544968714 +0000 UTC m=+71.973643896"
Jul 27 15:06:40.577435 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.577371    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc" podStartSLOduration=2.965125197 podStartE2EDuration="8.577347219s" totalImagesPullingTime="5.612222022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:06:40.575468532 +0000 UTC m=+72.004143716" watchObservedRunningTime="2026-07-27 15:06:40.577347219 +0000 UTC m=+72.006022404"
Jul 27 15:06:40.594851 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.594796    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7xbtq" podStartSLOduration=4.467830192 podStartE2EDuration="8.594775915s" totalImagesPullingTime="4.126945723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:06:40.593607875 +0000 UTC m=+72.022283059" watchObservedRunningTime="2026-07-27 15:06:40.594775915 +0000 UTC m=+72.023451101"
Jul 27 15:06:40.637359 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:40.637300    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r4tc2" podStartSLOduration=66.589765719 podStartE2EDuration="1m10.637278912s" totalImagesPullingTime="4.047513193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:06:40.612338906 +0000 UTC m=+72.041014089" watchObservedRunningTime="2026-07-27 15:06:40.637278912 +0000 UTC m=+72.065954097"
Jul 27 15:06:41.502169 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:41.502066    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x69pt" event={"ID":"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3","Type":"ContainerStarted","Data":"271c00d0dc09e92de1b1ba2ade556fefc8c2dca2f547995f8309adfe954beb78"}
Jul 27 15:06:41.502169 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:41.502133    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x69pt" event={"ID":"f21ffbbc-abb4-4ee7-8fbb-8780412fffc3","Type":"ContainerStarted","Data":"f4edeaf75d2aa6ce231c2346d628753a05b59ea6a46d9e6928d21deabc0de654"}
Jul 27 15:06:41.519886 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:41.519813    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-x69pt" podStartSLOduration=69.100759813 podStartE2EDuration="1m11.519791842s" totalImagesPullingTime="2.419032029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:06:41.519483289 +0000 UTC m=+72.948158473" watchObservedRunningTime="2026-07-27 15:06:41.519791842 +0000 UTC m=+72.948467027"
Jul 27 15:06:41.520931 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:41.520878    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c594d976d-4vgn5" podStartSLOduration=2.8903810659999998 podStartE2EDuration="8.520864004s" totalImagesPullingTime="5.630482938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:06:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:06:40.637805453 +0000 UTC m=+72.066480637" watchObservedRunningTime="2026-07-27 15:06:41.520864004 +0000 UTC m=+72.949539191"
Jul 27 15:06:41.644210 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:41.644176    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:41.744547 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:41.744498    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:41.849271 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:41.849192    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:42.509688 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:42.509603    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerStarted","Data":"0ce46a2a933a7eef1b5dcb144236671a1ecb4ad115adaa800cc8e95356bd64e7"}
Jul 27 15:06:42.511061 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:42.511032    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-njr4n" event={"ID":"f1023b4e-580a-4dc0-8a69-4167ae7cec34","Type":"ContainerStarted","Data":"4ddf61acbe487643eb4c5db41f45dda6ae13442d9023253d4ac29197310460b0"}
Jul 27 15:06:42.542278 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:42.542224    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.590213845 podStartE2EDuration="13.542204911s" totalImagesPullingTime="6.929548298s" totalInitContainerRuntime="22.442768ms" isStatefulPod=false podCreationTimestamp="2026-07-27 15:06:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:06:42.539835818 +0000 UTC m=+73.968511001" watchObservedRunningTime="2026-07-27 15:06:42.542204911 +0000 UTC m=+73.970880095"
Jul 27 15:06:42.559686 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:42.559638    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-njr4n" podStartSLOduration=66.576266707 podStartE2EDuration="1m12.559623713s" totalImagesPullingTime="5.983357006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 15:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:06:42.558768955 +0000 UTC m=+73.987444139" watchObservedRunningTime="2026-07-27 15:06:42.559623713 +0000 UTC m=+73.988298896"
Jul 27 15:06:43.029952 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:43.029919    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:43.140506 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:43.140468    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:43.281195 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:43.281104    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:43.372326 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:43.372292    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:43.372475 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:43.372350    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:43.377181 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:43.377162    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:43.385626 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:43.385604    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:43.518692 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:43.518663    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:06:43.745178 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:43.745150    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:43.846848 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:43.846820    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:43.955287 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:43.955250    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:45.131547 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:45.131510    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:45.290453 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:45.290414    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:45.733544 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:45.733478    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:45.862817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:45.862781    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:46.528811 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:46.528725    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerStarted","Data":"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"}
Jul 27 15:06:46.529293 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:46.528816    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerStarted","Data":"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"}
Jul 27 15:06:47.122385 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:47.122352    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:47.263115 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:47.263077    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:48.543623 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:48.543567    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerStarted","Data":"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"}
Jul 27 15:06:49.554200 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:49.554086    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerStarted","Data":"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"}
Jul 27 15:06:49.554200 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:49.554196    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerStarted","Data":"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"}
Jul 27 15:06:49.554623 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:49.554239    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerStarted","Data":"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"}
Jul 27 15:06:49.612037 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:49.611981    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.723145799 podStartE2EDuration="15.611963664s" totalImagesPullingTime="7.856501129s" totalInitContainerRuntime="32.316736ms" isStatefulPod=false podCreationTimestamp="2026-07-27 15:06:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:06:49.609115464 +0000 UTC m=+81.037790646" watchObservedRunningTime="2026-07-27 15:06:49.611963664 +0000 UTC m=+81.040638846"
Jul 27 15:06:49.837741 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:49.837651    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:06:52.936954 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:52.936924    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:52.937373 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:52.936983    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:06:53.151114 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:53.151077    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:53.249167 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:53.249132    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:54.545273 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:54.545240    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:06:54.648692 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:06:54.648664    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:07:02.122395 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:02.122364    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:07:02.217312 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:02.217273    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:07:06.004065 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:06.004030    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:07:08.147286 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:08.147246    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:07:12.513597 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:12.513563    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r4tc2"
Jul 27 15:07:12.942952 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:12.942881    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:07:12.946802 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:12.946781    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-9455d7f8-4lfdc"
Jul 27 15:07:34.838130 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:34.838094    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:34.858027 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:34.857997    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:35.738986 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:35.738960    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:47.531080 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:47.531046    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:07:47.891804 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:47.891719    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:07:48.593647 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.591808    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 15:07:48.593647 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.592336    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="alertmanager" containerID="cri-o://ce8b2f1276901b2d4a11fc73e862742dfe80161e60e7f6daf133c60f78663ab3" gracePeriod=120
Jul 27 15:07:48.593647 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.592706    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="kube-rbac-proxy" containerID="cri-o://f12948252e7849dc203c55a2254b6ae65bf957fe3b9ba41f1feb64e182c109be" gracePeriod=120
Jul 27 15:07:48.593647 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.592818    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="kube-rbac-proxy-web" containerID="cri-o://a9ff653e0b4a6b401e4affff90a8b08ea301cf197c12f489f00ceefa18e0af96" gracePeriod=120
Jul 27 15:07:48.593647 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.592891    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="prom-label-proxy" containerID="cri-o://0ce46a2a933a7eef1b5dcb144236671a1ecb4ad115adaa800cc8e95356bd64e7" gracePeriod=120
Jul 27 15:07:48.593647 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.592933    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="config-reloader" containerID="cri-o://0fd09f6dd661bc79d108df1d93d61b637e37bacf452522260a1eadcbfe142038" gracePeriod=120
Jul 27 15:07:48.593647 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.593012    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="kube-rbac-proxy-metric" containerID="cri-o://b8c0cd27101b891c6f7d353c6672a3dfd43621b969128942f623725cd6a8b410" gracePeriod=120
Jul 27 15:07:48.771845 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.771814    2548 generic.go:356] "Generic (PLEG): container finished" podID="52583d13-4407-4992-930e-84194a7aef1b" containerID="0ce46a2a933a7eef1b5dcb144236671a1ecb4ad115adaa800cc8e95356bd64e7" exitCode=0
Jul 27 15:07:48.771845 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.771839    2548 generic.go:356] "Generic (PLEG): container finished" podID="52583d13-4407-4992-930e-84194a7aef1b" containerID="f12948252e7849dc203c55a2254b6ae65bf957fe3b9ba41f1feb64e182c109be" exitCode=0
Jul 27 15:07:48.771845 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.771847    2548 generic.go:356] "Generic (PLEG): container finished" podID="52583d13-4407-4992-930e-84194a7aef1b" containerID="0fd09f6dd661bc79d108df1d93d61b637e37bacf452522260a1eadcbfe142038" exitCode=0
Jul 27 15:07:48.772053 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.771858    2548 generic.go:356] "Generic (PLEG): container finished" podID="52583d13-4407-4992-930e-84194a7aef1b" containerID="ce8b2f1276901b2d4a11fc73e862742dfe80161e60e7f6daf133c60f78663ab3" exitCode=0
Jul 27 15:07:48.772053 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.771891    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerDied","Data":"0ce46a2a933a7eef1b5dcb144236671a1ecb4ad115adaa800cc8e95356bd64e7"}
Jul 27 15:07:48.772053 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.771958    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerDied","Data":"f12948252e7849dc203c55a2254b6ae65bf957fe3b9ba41f1feb64e182c109be"}
Jul 27 15:07:48.772053 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.771985    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerDied","Data":"0fd09f6dd661bc79d108df1d93d61b637e37bacf452522260a1eadcbfe142038"}
Jul 27 15:07:48.772053 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:48.772010    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerDied","Data":"ce8b2f1276901b2d4a11fc73e862742dfe80161e60e7f6daf133c60f78663ab3"}
Jul 27 15:07:49.780288 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.780256    2548 generic.go:356] "Generic (PLEG): container finished" podID="52583d13-4407-4992-930e-84194a7aef1b" containerID="b8c0cd27101b891c6f7d353c6672a3dfd43621b969128942f623725cd6a8b410" exitCode=0
Jul 27 15:07:49.780288 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.780285    2548 generic.go:356] "Generic (PLEG): container finished" podID="52583d13-4407-4992-930e-84194a7aef1b" containerID="a9ff653e0b4a6b401e4affff90a8b08ea301cf197c12f489f00ceefa18e0af96" exitCode=0
Jul 27 15:07:49.780686 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.780329    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerDied","Data":"b8c0cd27101b891c6f7d353c6672a3dfd43621b969128942f623725cd6a8b410"}
Jul 27 15:07:49.780686 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.780403    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerDied","Data":"a9ff653e0b4a6b401e4affff90a8b08ea301cf197c12f489f00ceefa18e0af96"}
Jul 27 15:07:49.836278 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.836258    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:07:49.964031 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.963940    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-web-config\") pod \"52583d13-4407-4992-930e-84194a7aef1b\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") "
Jul 27 15:07:49.964031 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.963973    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-trusted-ca-bundle\") pod \"52583d13-4407-4992-930e-84194a7aef1b\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") "
Jul 27 15:07:49.964031 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964001    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-config-volume\") pod \"52583d13-4407-4992-930e-84194a7aef1b\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") "
Jul 27 15:07:49.964031 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964019    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-main-db\") pod \"52583d13-4407-4992-930e-84194a7aef1b\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") "
Jul 27 15:07:49.964371 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964048    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-cluster-tls-config\") pod \"52583d13-4407-4992-930e-84194a7aef1b\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") "
Jul 27 15:07:49.964371 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964086    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/52583d13-4407-4992-930e-84194a7aef1b-tls-assets\") pod \"52583d13-4407-4992-930e-84194a7aef1b\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") "
Jul 27 15:07:49.964371 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964102    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/52583d13-4407-4992-930e-84194a7aef1b-config-out\") pod \"52583d13-4407-4992-930e-84194a7aef1b\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") "
Jul 27 15:07:49.964371 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964118    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"52583d13-4407-4992-930e-84194a7aef1b\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") "
Jul 27 15:07:49.964371 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964137    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy-web\") pod \"52583d13-4407-4992-930e-84194a7aef1b\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") "
Jul 27 15:07:49.964371 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964163    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-metrics-client-ca\") pod \"52583d13-4407-4992-930e-84194a7aef1b\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") "
Jul 27 15:07:49.964371 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964193    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy\") pod \"52583d13-4407-4992-930e-84194a7aef1b\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") "
Jul 27 15:07:49.964371 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964245    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6mpb\" (UniqueName: \"kubernetes.io/projected/52583d13-4407-4992-930e-84194a7aef1b-kube-api-access-j6mpb\") pod \"52583d13-4407-4992-930e-84194a7aef1b\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") "
Jul 27 15:07:49.964371 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964280    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-main-tls\") pod \"52583d13-4407-4992-930e-84194a7aef1b\" (UID: \"52583d13-4407-4992-930e-84194a7aef1b\") "
Jul 27 15:07:49.964371 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964349    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "52583d13-4407-4992-930e-84194a7aef1b" (UID: "52583d13-4407-4992-930e-84194a7aef1b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 15:07:49.964918 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964404    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "52583d13-4407-4992-930e-84194a7aef1b" (UID: "52583d13-4407-4992-930e-84194a7aef1b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:07:49.964918 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964594    2548 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:49.964918 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.964617    2548 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/52583d13-4407-4992-930e-84194a7aef1b-alertmanager-main-db\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:49.965632 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.965589    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "52583d13-4407-4992-930e-84194a7aef1b" (UID: "52583d13-4407-4992-930e-84194a7aef1b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:07:49.967706 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.967635    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52583d13-4407-4992-930e-84194a7aef1b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "52583d13-4407-4992-930e-84194a7aef1b" (UID: "52583d13-4407-4992-930e-84194a7aef1b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:07:49.967706 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.967674    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "52583d13-4407-4992-930e-84194a7aef1b" (UID: "52583d13-4407-4992-930e-84194a7aef1b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:49.967931 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.967869    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52583d13-4407-4992-930e-84194a7aef1b-config-out" (OuterVolumeSpecName: "config-out") pod "52583d13-4407-4992-930e-84194a7aef1b" (UID: "52583d13-4407-4992-930e-84194a7aef1b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 15:07:49.967989 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.967968    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "52583d13-4407-4992-930e-84194a7aef1b" (UID: "52583d13-4407-4992-930e-84194a7aef1b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:49.968314 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.968283    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "52583d13-4407-4992-930e-84194a7aef1b" (UID: "52583d13-4407-4992-930e-84194a7aef1b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:49.968620 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.968597    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-config-volume" (OuterVolumeSpecName: "config-volume") pod "52583d13-4407-4992-930e-84194a7aef1b" (UID: "52583d13-4407-4992-930e-84194a7aef1b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:49.968963 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.968944    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "52583d13-4407-4992-930e-84194a7aef1b" (UID: "52583d13-4407-4992-930e-84194a7aef1b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:49.969194 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.969170    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52583d13-4407-4992-930e-84194a7aef1b-kube-api-access-j6mpb" (OuterVolumeSpecName: "kube-api-access-j6mpb") pod "52583d13-4407-4992-930e-84194a7aef1b" (UID: "52583d13-4407-4992-930e-84194a7aef1b"). InnerVolumeSpecName "kube-api-access-j6mpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:07:49.972608 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.972570    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "52583d13-4407-4992-930e-84194a7aef1b" (UID: "52583d13-4407-4992-930e-84194a7aef1b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:49.978703 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:49.978678    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-web-config" (OuterVolumeSpecName: "web-config") pod "52583d13-4407-4992-930e-84194a7aef1b" (UID: "52583d13-4407-4992-930e-84194a7aef1b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:50.065847 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.065812    2548 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-config-volume\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:50.065847 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.065840    2548 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-cluster-tls-config\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:50.065847 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.065849    2548 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/52583d13-4407-4992-930e-84194a7aef1b-tls-assets\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:50.066068 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.065857    2548 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/52583d13-4407-4992-930e-84194a7aef1b-config-out\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:50.066068 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.065868    2548 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:50.066068 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.065877    2548 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:50.066068 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.065887    2548 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52583d13-4407-4992-930e-84194a7aef1b-metrics-client-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:50.066068 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.065896    2548 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:50.066068 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.065904    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6mpb\" (UniqueName: \"kubernetes.io/projected/52583d13-4407-4992-930e-84194a7aef1b-kube-api-access-j6mpb\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:50.066068 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.065913    2548 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-secret-alertmanager-main-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:50.066068 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.065942    2548 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/52583d13-4407-4992-930e-84194a7aef1b-web-config\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:50.787381 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.787342    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52583d13-4407-4992-930e-84194a7aef1b","Type":"ContainerDied","Data":"593e0e33256f539290eed8532faa965a4591f7ab22ec1e1bf264491699d131fb"}
Jul 27 15:07:50.787381 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.787362    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 15:07:50.787872 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.787434    2548 scope.go:117] "RemoveContainer" containerID="0ce46a2a933a7eef1b5dcb144236671a1ecb4ad115adaa800cc8e95356bd64e7"
Jul 27 15:07:50.796269 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.796249    2548 scope.go:117] "RemoveContainer" containerID="b8c0cd27101b891c6f7d353c6672a3dfd43621b969128942f623725cd6a8b410"
Jul 27 15:07:50.803267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.803250    2548 scope.go:117] "RemoveContainer" containerID="f12948252e7849dc203c55a2254b6ae65bf957fe3b9ba41f1feb64e182c109be"
Jul 27 15:07:50.809695 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.809677    2548 scope.go:117] "RemoveContainer" containerID="a9ff653e0b4a6b401e4affff90a8b08ea301cf197c12f489f00ceefa18e0af96"
Jul 27 15:07:50.812930 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.812909    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 15:07:50.816444 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.816358    2548 scope.go:117] "RemoveContainer" containerID="0fd09f6dd661bc79d108df1d93d61b637e37bacf452522260a1eadcbfe142038"
Jul 27 15:07:50.818839 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.818819    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 15:07:50.823706 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.823689    2548 scope.go:117] "RemoveContainer" containerID="ce8b2f1276901b2d4a11fc73e862742dfe80161e60e7f6daf133c60f78663ab3"
Jul 27 15:07:50.830237 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:50.830222    2548 scope.go:117] "RemoveContainer" containerID="10dd7ac226b368be5b0f70d4eb56823a5a30e1e9defc9c72e383ee0c60985130"
Jul 27 15:07:51.137650 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:51.137616    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52583d13-4407-4992-930e-84194a7aef1b" path="/var/lib/kubelet/pods/52583d13-4407-4992-930e-84194a7aef1b/volumes"
Jul 27 15:07:52.631298 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631265    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"]
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631511    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="config-reloader"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631522    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="config-reloader"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631553    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="kube-rbac-proxy"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631559    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="kube-rbac-proxy"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631566    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="alertmanager"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631572    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="alertmanager"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631579    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="init-config-reloader"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631584    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="init-config-reloader"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631605    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="kube-rbac-proxy-metric"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631611    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="kube-rbac-proxy-metric"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631622    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="kube-rbac-proxy-web"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631628    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="kube-rbac-proxy-web"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631640    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="prom-label-proxy"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631645    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="prom-label-proxy"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631682    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="kube-rbac-proxy-metric"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631689    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="prom-label-proxy"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631696    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="kube-rbac-proxy"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631702    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="kube-rbac-proxy-web"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631707    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="alertmanager"
Jul 27 15:07:52.631791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.631712    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="52583d13-4407-4992-930e-84194a7aef1b" containerName="config-reloader"
Jul 27 15:07:52.634734 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.634714    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.637686 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.637666    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 27 15:07:52.637686 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.637676    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 27 15:07:52.637978 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.637957    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 27 15:07:52.638055 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.638038    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 27 15:07:52.638114 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.638058    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-4l6qr\""
Jul 27 15:07:52.638167 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.638140    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 27 15:07:52.642923 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.642891    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 27 15:07:52.647454 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.647434    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"]
Jul 27 15:07:52.789655 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.789626    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/12742c2c-05be-4268-a2f6-e367af8aa791-telemeter-client-tls\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.789655 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.789660    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/12742c2c-05be-4268-a2f6-e367af8aa791-federate-client-tls\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.789892 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.789680    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12742c2c-05be-4268-a2f6-e367af8aa791-serving-certs-ca-bundle\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.789892 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.789709    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12742c2c-05be-4268-a2f6-e367af8aa791-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.789892 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.789748    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4cjn\" (UniqueName: \"kubernetes.io/projected/12742c2c-05be-4268-a2f6-e367af8aa791-kube-api-access-k4cjn\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.789892 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.789767    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/12742c2c-05be-4268-a2f6-e367af8aa791-secret-telemeter-client\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.789892 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.789786    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12742c2c-05be-4268-a2f6-e367af8aa791-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.789892 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.789870    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12742c2c-05be-4268-a2f6-e367af8aa791-metrics-client-ca\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.891174 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.891061    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/12742c2c-05be-4268-a2f6-e367af8aa791-secret-telemeter-client\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.891174 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.891117    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12742c2c-05be-4268-a2f6-e367af8aa791-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.891174 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.891147    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12742c2c-05be-4268-a2f6-e367af8aa791-metrics-client-ca\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.891174 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.891173    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/12742c2c-05be-4268-a2f6-e367af8aa791-telemeter-client-tls\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.891497 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.891193    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/12742c2c-05be-4268-a2f6-e367af8aa791-federate-client-tls\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.891497 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.891210    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12742c2c-05be-4268-a2f6-e367af8aa791-serving-certs-ca-bundle\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.891497 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.891247    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12742c2c-05be-4268-a2f6-e367af8aa791-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.891497 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.891284    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4cjn\" (UniqueName: \"kubernetes.io/projected/12742c2c-05be-4268-a2f6-e367af8aa791-kube-api-access-k4cjn\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.892155 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.892081    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12742c2c-05be-4268-a2f6-e367af8aa791-metrics-client-ca\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.892269 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.892180    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12742c2c-05be-4268-a2f6-e367af8aa791-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.892269 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.892229    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12742c2c-05be-4268-a2f6-e367af8aa791-serving-certs-ca-bundle\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.893829 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.893800    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/12742c2c-05be-4268-a2f6-e367af8aa791-federate-client-tls\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.893932 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.893908    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12742c2c-05be-4268-a2f6-e367af8aa791-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.894177 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.894161    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/12742c2c-05be-4268-a2f6-e367af8aa791-telemeter-client-tls\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.894371 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.894354    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/12742c2c-05be-4268-a2f6-e367af8aa791-secret-telemeter-client\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.899245 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.899229    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4cjn\" (UniqueName: \"kubernetes.io/projected/12742c2c-05be-4268-a2f6-e367af8aa791-kube-api-access-k4cjn\") pod \"telemeter-client-85b764cd7f-8cfkc\" (UID: \"12742c2c-05be-4268-a2f6-e367af8aa791\") " pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.944972 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.944939    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"
Jul 27 15:07:52.948791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.948767    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 15:07:52.949213 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.949029    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="prometheus" containerID="cri-o://57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c" gracePeriod=600
Jul 27 15:07:52.949213 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.949073    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="thanos-sidecar" containerID="cri-o://96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b" gracePeriod=600
Jul 27 15:07:52.949213 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.949074    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="kube-rbac-proxy" containerID="cri-o://b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930" gracePeriod=600
Jul 27 15:07:52.949213 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.949095    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="config-reloader" containerID="cri-o://f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0" gracePeriod=600
Jul 27 15:07:52.949213 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.949150    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="kube-rbac-proxy-thanos" containerID="cri-o://2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823" gracePeriod=600
Jul 27 15:07:52.949498 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:52.949260    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="kube-rbac-proxy-web" containerID="cri-o://cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37" gracePeriod=600
Jul 27 15:07:53.093146 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.093111    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85b764cd7f-8cfkc"]
Jul 27 15:07:53.096362 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:07:53.096333    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12742c2c_05be_4268_a2f6_e367af8aa791.slice/crio-09f2b9a0921e091c4b04971cf8585ecf09fa8717ec568bc42a5905ba915cb891 WatchSource:0}: Error finding container 09f2b9a0921e091c4b04971cf8585ecf09fa8717ec568bc42a5905ba915cb891: Status 404 returned error can't find the container with id 09f2b9a0921e091c4b04971cf8585ecf09fa8717ec568bc42a5905ba915cb891
Jul 27 15:07:53.190996 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.190974    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:53.295193 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295164    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-thanos-prometheus-http-client-file\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295193 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295198    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-2\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295386 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295214    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-metrics-client-ca\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295386 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295240    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-1\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295386 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295271    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2ced953-3363-44c8-8bcb-2684fa915e40-config-out\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295386 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295296    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-config\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295386 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295317    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-metrics-client-certs\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295386 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295342    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-web-config\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295386 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295372    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-kube-rbac-proxy\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295748 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295407    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-grpc-tls\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295748 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295439    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-serving-certs-ca-bundle\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295748 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295467    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-db\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295748 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295496    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-kubelet-serving-ca-bundle\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295748 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295547    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-tls\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295748 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295586    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295748 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295619    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2ced953-3363-44c8-8bcb-2684fa915e40-tls-assets\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295748 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295647    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mck8k\" (UniqueName: \"kubernetes.io/projected/d2ced953-3363-44c8-8bcb-2684fa915e40-kube-api-access-mck8k\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295748 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295684    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.295748 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295731    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-trusted-ca-bundle\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.296210 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295760    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-0\") pod \"d2ced953-3363-44c8-8bcb-2684fa915e40\" (UID: \"d2ced953-3363-44c8-8bcb-2684fa915e40\") "
Jul 27 15:07:53.299100 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295637    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:07:53.299216 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295650    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:07:53.299216 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.295709    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:07:53.299216 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.296139    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:07:53.299216 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.296779    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:07:53.299216 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.299059    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 15:07:53.299643 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.299252    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2ced953-3363-44c8-8bcb-2684fa915e40-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:07:53.299643 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.299334    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:53.299643 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.299492    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:53.299811 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.299654    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:53.299869 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.299806    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:53.300138 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.300109    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:07:53.300502 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.300463    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:53.300502 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.300491    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-config" (OuterVolumeSpecName: "config") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:53.301043 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.301022    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:53.301171 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.301153    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2ced953-3363-44c8-8bcb-2684fa915e40-kube-api-access-mck8k" (OuterVolumeSpecName: "kube-api-access-mck8k") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "kube-api-access-mck8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:07:53.301727 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.301703    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:07:53.301807 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.301791    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2ced953-3363-44c8-8bcb-2684fa915e40-config-out" (OuterVolumeSpecName: "config-out") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 15:07:53.302122 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.302107    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:53.311355 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.311329    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-web-config" (OuterVolumeSpecName: "web-config") pod "d2ced953-3363-44c8-8bcb-2684fa915e40" (UID: "d2ced953-3363-44c8-8bcb-2684fa915e40"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:07:53.396463 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396438    2548 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396463 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396463    2548 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396472    2548 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2ced953-3363-44c8-8bcb-2684fa915e40-tls-assets\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396482    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mck8k\" (UniqueName: \"kubernetes.io/projected/d2ced953-3363-44c8-8bcb-2684fa915e40-kube-api-access-mck8k\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396490    2548 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396499    2548 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396508    2548 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396516    2548 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396525    2548 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396550    2548 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-metrics-client-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396557    2548 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396565    2548 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2ced953-3363-44c8-8bcb-2684fa915e40-config-out\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396573    2548 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-config\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396580    2548 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-metrics-client-certs\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396588    2548 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-web-config\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396595    2548 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-kube-rbac-proxy\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396604    2548 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2ced953-3363-44c8-8bcb-2684fa915e40-secret-grpc-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396613    2548 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396620    2548 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2ced953-3363-44c8-8bcb-2684fa915e40-prometheus-k8s-db\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.396624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.396628    2548 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2ced953-3363-44c8-8bcb-2684fa915e40-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:07:53.804389 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804347    2548 generic.go:356] "Generic (PLEG): container finished" podID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerID="2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823" exitCode=0
Jul 27 15:07:53.804389 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804382    2548 generic.go:356] "Generic (PLEG): container finished" podID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerID="b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930" exitCode=0
Jul 27 15:07:53.804389 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804394    2548 generic.go:356] "Generic (PLEG): container finished" podID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerID="cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37" exitCode=0
Jul 27 15:07:53.804933 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804406    2548 generic.go:356] "Generic (PLEG): container finished" podID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerID="96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b" exitCode=0
Jul 27 15:07:53.804933 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804415    2548 generic.go:356] "Generic (PLEG): container finished" podID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerID="f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0" exitCode=0
Jul 27 15:07:53.804933 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804424    2548 generic.go:356] "Generic (PLEG): container finished" podID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerID="57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c" exitCode=0
Jul 27 15:07:53.804933 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804437    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:53.804933 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804432    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerDied","Data":"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"}
Jul 27 15:07:53.804933 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804586    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerDied","Data":"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"}
Jul 27 15:07:53.804933 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804635    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerDied","Data":"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"}
Jul 27 15:07:53.804933 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804677    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerDied","Data":"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"}
Jul 27 15:07:53.804933 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804657    2548 scope.go:117] "RemoveContainer" containerID="2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"
Jul 27 15:07:53.804933 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804717    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerDied","Data":"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"}
Jul 27 15:07:53.804933 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804757    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerDied","Data":"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"}
Jul 27 15:07:53.804933 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.804791    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2ced953-3363-44c8-8bcb-2684fa915e40","Type":"ContainerDied","Data":"0692eba3d1d77378f79d45d4a29bee27dedd4b60b4655467679b413d8beab1f0"}
Jul 27 15:07:53.806523 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.806489    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc" event={"ID":"12742c2c-05be-4268-a2f6-e367af8aa791","Type":"ContainerStarted","Data":"09f2b9a0921e091c4b04971cf8585ecf09fa8717ec568bc42a5905ba915cb891"}
Jul 27 15:07:53.814663 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.814369    2548 scope.go:117] "RemoveContainer" containerID="b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"
Jul 27 15:07:53.821984 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.821965    2548 scope.go:117] "RemoveContainer" containerID="cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"
Jul 27 15:07:53.829722 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.829703    2548 scope.go:117] "RemoveContainer" containerID="96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"
Jul 27 15:07:53.836270 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.836242    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 15:07:53.838031 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.838013    2548 scope.go:117] "RemoveContainer" containerID="f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"
Jul 27 15:07:53.838788 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.838768    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 15:07:53.845100 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.845080    2548 scope.go:117] "RemoveContainer" containerID="57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"
Jul 27 15:07:53.855275 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.855252    2548 scope.go:117] "RemoveContainer" containerID="367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"
Jul 27 15:07:53.863102 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.863081    2548 scope.go:117] "RemoveContainer" containerID="2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"
Jul 27 15:07:53.863366 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:07:53.863335    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823\": container with ID starting with 2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823 not found: ID does not exist" containerID="2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"
Jul 27 15:07:53.863449 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.863368    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"} err="failed to get container status \"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823\": rpc error: code = NotFound desc = could not find container \"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823\": container with ID starting with 2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823 not found: ID does not exist"
Jul 27 15:07:53.863449 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.863411    2548 scope.go:117] "RemoveContainer" containerID="b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"
Jul 27 15:07:53.863674 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:07:53.863650    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930\": container with ID starting with b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930 not found: ID does not exist" containerID="b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"
Jul 27 15:07:53.863767 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.863679    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"} err="failed to get container status \"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930\": rpc error: code = NotFound desc = could not find container \"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930\": container with ID starting with b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930 not found: ID does not exist"
Jul 27 15:07:53.863767 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.863697    2548 scope.go:117] "RemoveContainer" containerID="cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"
Jul 27 15:07:53.863963 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:07:53.863939    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37\": container with ID starting with cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37 not found: ID does not exist" containerID="cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"
Jul 27 15:07:53.864017 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.863971    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"} err="failed to get container status \"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37\": rpc error: code = NotFound desc = could not find container \"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37\": container with ID starting with cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37 not found: ID does not exist"
Jul 27 15:07:53.864017 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.863993    2548 scope.go:117] "RemoveContainer" containerID="96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"
Jul 27 15:07:53.864256 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:07:53.864234    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b\": container with ID starting with 96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b not found: ID does not exist" containerID="96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"
Jul 27 15:07:53.864327 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.864261    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"} err="failed to get container status \"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b\": rpc error: code = NotFound desc = could not find container \"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b\": container with ID starting with 96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b not found: ID does not exist"
Jul 27 15:07:53.864327 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.864278    2548 scope.go:117] "RemoveContainer" containerID="f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"
Jul 27 15:07:53.864599 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:07:53.864506    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0\": container with ID starting with f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0 not found: ID does not exist" containerID="f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"
Jul 27 15:07:53.864599 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.864554    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"} err="failed to get container status \"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0\": rpc error: code = NotFound desc = could not find container \"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0\": container with ID starting with f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0 not found: ID does not exist"
Jul 27 15:07:53.864599 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.864571    2548 scope.go:117] "RemoveContainer" containerID="57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"
Jul 27 15:07:53.864811 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:07:53.864789    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c\": container with ID starting with 57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c not found: ID does not exist" containerID="57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"
Jul 27 15:07:53.864861 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.864820    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"} err="failed to get container status \"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c\": rpc error: code = NotFound desc = could not find container \"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c\": container with ID starting with 57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c not found: ID does not exist"
Jul 27 15:07:53.864861 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.864839    2548 scope.go:117] "RemoveContainer" containerID="367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"
Jul 27 15:07:53.865099 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:07:53.865078    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383\": container with ID starting with 367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383 not found: ID does not exist" containerID="367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"
Jul 27 15:07:53.865201 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.865100    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"} err="failed to get container status \"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383\": rpc error: code = NotFound desc = could not find container \"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383\": container with ID starting with 367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383 not found: ID does not exist"
Jul 27 15:07:53.865201 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.865112    2548 scope.go:117] "RemoveContainer" containerID="2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"
Jul 27 15:07:53.865365 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.865343    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"} err="failed to get container status \"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823\": rpc error: code = NotFound desc = could not find container \"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823\": container with ID starting with 2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823 not found: ID does not exist"
Jul 27 15:07:53.865427 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.865367    2548 scope.go:117] "RemoveContainer" containerID="b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"
Jul 27 15:07:53.865654 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.865633    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"} err="failed to get container status \"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930\": rpc error: code = NotFound desc = could not find container \"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930\": container with ID starting with b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930 not found: ID does not exist"
Jul 27 15:07:53.865779 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.865661    2548 scope.go:117] "RemoveContainer" containerID="cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"
Jul 27 15:07:53.865926 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.865901    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"} err="failed to get container status \"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37\": rpc error: code = NotFound desc = could not find container \"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37\": container with ID starting with cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37 not found: ID does not exist"
Jul 27 15:07:53.866011 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.865931    2548 scope.go:117] "RemoveContainer" containerID="96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"
Jul 27 15:07:53.866213 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.866191    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"} err="failed to get container status \"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b\": rpc error: code = NotFound desc = could not find container \"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b\": container with ID starting with 96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b not found: ID does not exist"
Jul 27 15:07:53.866213 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.866213    2548 scope.go:117] "RemoveContainer" containerID="f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"
Jul 27 15:07:53.866445 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.866426    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"} err="failed to get container status \"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0\": rpc error: code = NotFound desc = could not find container \"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0\": container with ID starting with f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0 not found: ID does not exist"
Jul 27 15:07:53.866505 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.866447    2548 scope.go:117] "RemoveContainer" containerID="57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"
Jul 27 15:07:53.866781 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.866755    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"} err="failed to get container status \"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c\": rpc error: code = NotFound desc = could not find container \"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c\": container with ID starting with 57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c not found: ID does not exist"
Jul 27 15:07:53.866859 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.866783    2548 scope.go:117] "RemoveContainer" containerID="367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"
Jul 27 15:07:53.867100 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.867079    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"} err="failed to get container status \"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383\": rpc error: code = NotFound desc = could not find container \"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383\": container with ID starting with 367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383 not found: ID does not exist"
Jul 27 15:07:53.867172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.867101    2548 scope.go:117] "RemoveContainer" containerID="2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"
Jul 27 15:07:53.867344 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.867321    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"} err="failed to get container status \"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823\": rpc error: code = NotFound desc = could not find container \"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823\": container with ID starting with 2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823 not found: ID does not exist"
Jul 27 15:07:53.867420 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.867347    2548 scope.go:117] "RemoveContainer" containerID="b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"
Jul 27 15:07:53.867613 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.867585    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"} err="failed to get container status \"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930\": rpc error: code = NotFound desc = could not find container \"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930\": container with ID starting with b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930 not found: ID does not exist"
Jul 27 15:07:53.867754 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.867616    2548 scope.go:117] "RemoveContainer" containerID="cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"
Jul 27 15:07:53.867875 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.867854    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"} err="failed to get container status \"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37\": rpc error: code = NotFound desc = could not find container \"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37\": container with ID starting with cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37 not found: ID does not exist"
Jul 27 15:07:53.868028 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.867886    2548 scope.go:117] "RemoveContainer" containerID="96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"
Jul 27 15:07:53.868167 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.868139    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"} err="failed to get container status \"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b\": rpc error: code = NotFound desc = could not find container \"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b\": container with ID starting with 96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b not found: ID does not exist"
Jul 27 15:07:53.868167 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.868168    2548 scope.go:117] "RemoveContainer" containerID="f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"
Jul 27 15:07:53.868546 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.868424    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"} err="failed to get container status \"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0\": rpc error: code = NotFound desc = could not find container \"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0\": container with ID starting with f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0 not found: ID does not exist"
Jul 27 15:07:53.868546 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.868466    2548 scope.go:117] "RemoveContainer" containerID="57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"
Jul 27 15:07:53.868791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.868759    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"} err="failed to get container status \"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c\": rpc error: code = NotFound desc = could not find container \"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c\": container with ID starting with 57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c not found: ID does not exist"
Jul 27 15:07:53.868875 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.868794    2548 scope.go:117] "RemoveContainer" containerID="367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"
Jul 27 15:07:53.869050 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.869028    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"} err="failed to get container status \"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383\": rpc error: code = NotFound desc = could not find container \"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383\": container with ID starting with 367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383 not found: ID does not exist"
Jul 27 15:07:53.869124 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.869054    2548 scope.go:117] "RemoveContainer" containerID="2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"
Jul 27 15:07:53.869300 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.869282    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"} err="failed to get container status \"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823\": rpc error: code = NotFound desc = could not find container \"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823\": container with ID starting with 2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823 not found: ID does not exist"
Jul 27 15:07:53.869385 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.869303    2548 scope.go:117] "RemoveContainer" containerID="b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"
Jul 27 15:07:53.869565 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.869525    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"} err="failed to get container status \"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930\": rpc error: code = NotFound desc = could not find container \"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930\": container with ID starting with b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930 not found: ID does not exist"
Jul 27 15:07:53.869635 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.869566    2548 scope.go:117] "RemoveContainer" containerID="cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"
Jul 27 15:07:53.869798 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.869779    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"} err="failed to get container status \"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37\": rpc error: code = NotFound desc = could not find container \"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37\": container with ID starting with cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37 not found: ID does not exist"
Jul 27 15:07:53.869867 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.869800    2548 scope.go:117] "RemoveContainer" containerID="96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"
Jul 27 15:07:53.870051 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870032    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"} err="failed to get container status \"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b\": rpc error: code = NotFound desc = could not find container \"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b\": container with ID starting with 96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b not found: ID does not exist"
Jul 27 15:07:53.870134 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870053    2548 scope.go:117] "RemoveContainer" containerID="f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"
Jul 27 15:07:53.870320 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870298    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"} err="failed to get container status \"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0\": rpc error: code = NotFound desc = could not find container \"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0\": container with ID starting with f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0 not found: ID does not exist"
Jul 27 15:07:53.870370 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870322    2548 scope.go:117] "RemoveContainer" containerID="57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"
Jul 27 15:07:53.870495 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870480    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 15:07:53.870581 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870493    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"} err="failed to get container status \"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c\": rpc error: code = NotFound desc = could not find container \"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c\": container with ID starting with 57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c not found: ID does not exist"
Jul 27 15:07:53.870581 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870513    2548 scope.go:117] "RemoveContainer" containerID="367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"
Jul 27 15:07:53.870764 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870744    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"} err="failed to get container status \"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383\": rpc error: code = NotFound desc = could not find container \"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383\": container with ID starting with 367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383 not found: ID does not exist"
Jul 27 15:07:53.870764 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870763    2548 scope.go:117] "RemoveContainer" containerID="2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"
Jul 27 15:07:53.870888 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870808    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="prometheus"
Jul 27 15:07:53.870888 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870838    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="prometheus"
Jul 27 15:07:53.870888 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870848    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="config-reloader"
Jul 27 15:07:53.870888 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870857    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="config-reloader"
Jul 27 15:07:53.870888 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870869    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="kube-rbac-proxy-web"
Jul 27 15:07:53.870888 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870875    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="kube-rbac-proxy-web"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870890    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="init-config-reloader"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870897    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="init-config-reloader"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870911    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="thanos-sidecar"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870917    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="thanos-sidecar"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870925    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="kube-rbac-proxy"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870930    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="kube-rbac-proxy"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870942    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="kube-rbac-proxy-thanos"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870951    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="kube-rbac-proxy-thanos"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.870989    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"} err="failed to get container status \"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823\": rpc error: code = NotFound desc = could not find container \"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823\": container with ID starting with 2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823 not found: ID does not exist"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.871008    2548 scope.go:117] "RemoveContainer" containerID="b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.871010    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="config-reloader"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.871057    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="prometheus"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.871063    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="kube-rbac-proxy"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.871070    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="kube-rbac-proxy-web"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.871078    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="kube-rbac-proxy-thanos"
Jul 27 15:07:53.871126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.871088    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" containerName="thanos-sidecar"
Jul 27 15:07:53.871791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.871267    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"} err="failed to get container status \"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930\": rpc error: code = NotFound desc = could not find container \"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930\": container with ID starting with b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930 not found: ID does not exist"
Jul 27 15:07:53.871791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.871290    2548 scope.go:117] "RemoveContainer" containerID="cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"
Jul 27 15:07:53.871791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.871554    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"} err="failed to get container status \"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37\": rpc error: code = NotFound desc = could not find container \"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37\": container with ID starting with cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37 not found: ID does not exist"
Jul 27 15:07:53.871791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.871575    2548 scope.go:117] "RemoveContainer" containerID="96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"
Jul 27 15:07:53.871791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.871781    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"} err="failed to get container status \"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b\": rpc error: code = NotFound desc = could not find container \"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b\": container with ID starting with 96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b not found: ID does not exist"
Jul 27 15:07:53.872042 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.871799    2548 scope.go:117] "RemoveContainer" containerID="f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"
Jul 27 15:07:53.872133 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.872109    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"} err="failed to get container status \"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0\": rpc error: code = NotFound desc = could not find container \"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0\": container with ID starting with f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0 not found: ID does not exist"
Jul 27 15:07:53.872197 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.872136    2548 scope.go:117] "RemoveContainer" containerID="57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"
Jul 27 15:07:53.872404 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.872347    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"} err="failed to get container status \"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c\": rpc error: code = NotFound desc = could not find container \"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c\": container with ID starting with 57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c not found: ID does not exist"
Jul 27 15:07:53.872404 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.872370    2548 scope.go:117] "RemoveContainer" containerID="367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"
Jul 27 15:07:53.872663 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.872627    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"} err="failed to get container status \"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383\": rpc error: code = NotFound desc = could not find container \"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383\": container with ID starting with 367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383 not found: ID does not exist"
Jul 27 15:07:53.872663 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.872649    2548 scope.go:117] "RemoveContainer" containerID="2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"
Jul 27 15:07:53.873177 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.873158    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823"} err="failed to get container status \"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823\": rpc error: code = NotFound desc = could not find container \"2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823\": container with ID starting with 2d9fcd04456d7662de123c25cad99ec67c0e0f791d6f0ef6271cb60b0ceff823 not found: ID does not exist"
Jul 27 15:07:53.873177 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.873176    2548 scope.go:117] "RemoveContainer" containerID="b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"
Jul 27 15:07:53.873466 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.873446    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930"} err="failed to get container status \"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930\": rpc error: code = NotFound desc = could not find container \"b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930\": container with ID starting with b82078041dc338a413cc229598f3afe78a548883dda9f2d4d3839e0f80f9c930 not found: ID does not exist"
Jul 27 15:07:53.873650 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.873467    2548 scope.go:117] "RemoveContainer" containerID="cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"
Jul 27 15:07:53.873797 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.873773    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37"} err="failed to get container status \"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37\": rpc error: code = NotFound desc = could not find container \"cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37\": container with ID starting with cbc013a719fd889218b0ba69f4c5c405e50f92d8bbd6c25a3585809c89c52f37 not found: ID does not exist"
Jul 27 15:07:53.873865 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.873799    2548 scope.go:117] "RemoveContainer" containerID="96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"
Jul 27 15:07:53.874138 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.874113    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b"} err="failed to get container status \"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b\": rpc error: code = NotFound desc = could not find container \"96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b\": container with ID starting with 96efc2c79b0f91b7a2ec6160283e9cb169d572f658dc5a5a3a44a12503ea7b8b not found: ID does not exist"
Jul 27 15:07:53.874138 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.874135    2548 scope.go:117] "RemoveContainer" containerID="f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"
Jul 27 15:07:53.874356 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.874326    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0"} err="failed to get container status \"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0\": rpc error: code = NotFound desc = could not find container \"f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0\": container with ID starting with f827455ecfabf518bede4ba0045c2b214910c80a61478931b65fb50787bd04d0 not found: ID does not exist"
Jul 27 15:07:53.874356 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.874348    2548 scope.go:117] "RemoveContainer" containerID="57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"
Jul 27 15:07:53.874599 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.874574    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:53.874599 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.874586    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c"} err="failed to get container status \"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c\": rpc error: code = NotFound desc = could not find container \"57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c\": container with ID starting with 57a2c8d781085017914c5611b326cbe0f37592994597f5c725ad01c7a465776c not found: ID does not exist"
Jul 27 15:07:53.874745 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.874609    2548 scope.go:117] "RemoveContainer" containerID="367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"
Jul 27 15:07:53.874875 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.874848    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383"} err="failed to get container status \"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383\": rpc error: code = NotFound desc = could not find container \"367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383\": container with ID starting with 367eeada9ced679f8252a0f72c63c4babd4ab1a40d22a81b9c633d6c8d266383 not found: ID does not exist"
Jul 27 15:07:53.877344 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.877324    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 27 15:07:53.877480 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.877463    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zpbrk\""
Jul 27 15:07:53.877619 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.877603    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 27 15:07:53.877771 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.877752    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 27 15:07:53.877889 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.877872    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 27 15:07:53.877956 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.877904    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 27 15:07:53.877956 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.877949    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 27 15:07:53.878060 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.877984    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 27 15:07:53.878060 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.877903    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 27 15:07:53.878139 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.878094    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 27 15:07:53.878281 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.878268    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 27 15:07:53.878449 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.878438    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8npb9fcu135uc\""
Jul 27 15:07:53.878489 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.878474    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 27 15:07:53.878705 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.878692    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 27 15:07:53.881172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.881036    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 27 15:07:53.883316 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.883292    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 27 15:07:53.892617 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:53.892595    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 15:07:54.002545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002498    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.002693 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002547    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-config\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.002693 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002591    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ad3e603-6239-4042-ae14-8a788a6c757a-config-out\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.002693 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002607    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4nbx\" (UniqueName: \"kubernetes.io/projected/1ad3e603-6239-4042-ae14-8a788a6c757a-kube-api-access-m4nbx\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.002693 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002622    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.002693 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002653    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.002693 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002674    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.002693 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002692    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.002899 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002726    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.002899 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002757    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.002899 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002802    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.002899 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002817    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.002899 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002831    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ad3e603-6239-4042-ae14-8a788a6c757a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.002899 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002847    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.002899 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002860    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.003130 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002918    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.003130 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002939    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.003130 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.002980    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-web-config\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.003130 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.003006    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.003130 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.003024    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.103629 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.103557    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.103629 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.103594    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.103629 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.103625    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.103872 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.103674    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.103872 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.103700    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.103872 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.103736    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.103872 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.103752    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104037 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.103875    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ad3e603-6239-4042-ae14-8a788a6c757a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104037 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.103905    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104037 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.103939    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104037 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.103973    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104037 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.103982    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104037 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.103992    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104037 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.104020    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-web-config\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104390 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.104050    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104390 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.104079    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104390 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.104106    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104390 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.104142    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-config\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104390 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.104165    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ad3e603-6239-4042-ae14-8a788a6c757a-config-out\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104390 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.104188    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4nbx\" (UniqueName: \"kubernetes.io/projected/1ad3e603-6239-4042-ae14-8a788a6c757a-kube-api-access-m4nbx\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104390 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.104212    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.104910 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.104843    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.105413 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.105081    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.105668 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.105644    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.108194 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.105917    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.108194 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.106338    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.108194 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.106479    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.108194 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.107478    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.108194 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.107688    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.108194 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.107931    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.108725 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.108449    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.108834 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.108812    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-config\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.109290 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.109243    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.109374 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.109324    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.109666 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.109649    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ad3e603-6239-4042-ae14-8a788a6c757a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.109888 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.109867    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ad3e603-6239-4042-ae14-8a788a6c757a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.110156 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.110139    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.110201 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.110182    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ad3e603-6239-4042-ae14-8a788a6c757a-config-out\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.110802 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.110786    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ad3e603-6239-4042-ae14-8a788a6c757a-web-config\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.113839 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.113819    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4nbx\" (UniqueName: \"kubernetes.io/projected/1ad3e603-6239-4042-ae14-8a788a6c757a-kube-api-access-m4nbx\") pod \"prometheus-k8s-0\" (UID: \"1ad3e603-6239-4042-ae14-8a788a6c757a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.186210 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.186175    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:07:54.327762 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.327740    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 15:07:54.329984 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:07:54.329959    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ad3e603_6239_4042_ae14_8a788a6c757a.slice/crio-3ee489a7ee1db3aa97793a4aba26cbf16eea2da0fd62127c05aa0a0ceb0758c0 WatchSource:0}: Error finding container 3ee489a7ee1db3aa97793a4aba26cbf16eea2da0fd62127c05aa0a0ceb0758c0: Status 404 returned error can't find the container with id 3ee489a7ee1db3aa97793a4aba26cbf16eea2da0fd62127c05aa0a0ceb0758c0
Jul 27 15:07:54.814298 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.814264    2548 generic.go:356] "Generic (PLEG): container finished" podID="1ad3e603-6239-4042-ae14-8a788a6c757a" containerID="c415b00ad1362584d440dbedd6ff2f19193b2d47bb5c60ece545abf191e1d78e" exitCode=0
Jul 27 15:07:54.814725 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.814307    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ad3e603-6239-4042-ae14-8a788a6c757a","Type":"ContainerDied","Data":"c415b00ad1362584d440dbedd6ff2f19193b2d47bb5c60ece545abf191e1d78e"}
Jul 27 15:07:54.814725 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:54.814373    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ad3e603-6239-4042-ae14-8a788a6c757a","Type":"ContainerStarted","Data":"3ee489a7ee1db3aa97793a4aba26cbf16eea2da0fd62127c05aa0a0ceb0758c0"}
Jul 27 15:07:55.137220 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:55.137193    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2ced953-3363-44c8-8bcb-2684fa915e40" path="/var/lib/kubelet/pods/d2ced953-3363-44c8-8bcb-2684fa915e40/volumes"
Jul 27 15:07:55.821916 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:55.821877    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ad3e603-6239-4042-ae14-8a788a6c757a","Type":"ContainerStarted","Data":"470892f3e3482e20d60b580a72ebf18fc2327dd8dd2c60f238aca4f8a396fce9"}
Jul 27 15:07:55.822330 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:55.821964    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ad3e603-6239-4042-ae14-8a788a6c757a","Type":"ContainerStarted","Data":"5d34761fafc44572ca2b14e95291b8911617a26f24fd405a58fb37c9ccf5b435"}
Jul 27 15:07:55.822330 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:55.822007    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ad3e603-6239-4042-ae14-8a788a6c757a","Type":"ContainerStarted","Data":"227509fd790028e358c9c3b8aaf200314778094186ad1b88f4b592d2de17b50d"}
Jul 27 15:07:55.822330 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:55.822034    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ad3e603-6239-4042-ae14-8a788a6c757a","Type":"ContainerStarted","Data":"ebeae366631a4da759cca7b4ded16ce4630177ed76d130d6fe72838b1bb6bc09"}
Jul 27 15:07:55.822330 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:55.822065    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ad3e603-6239-4042-ae14-8a788a6c757a","Type":"ContainerStarted","Data":"5596cab8d66f588c03bea21b05840ff0705b869b2545893d7beeec039498ca50"}
Jul 27 15:07:55.822330 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:55.822095    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ad3e603-6239-4042-ae14-8a788a6c757a","Type":"ContainerStarted","Data":"b4fb95132b62199e2c6debba918da48bb0e9bee9bf0bdf9e6844300e286b425b"}
Jul 27 15:07:55.823996 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:55.823961    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc" event={"ID":"12742c2c-05be-4268-a2f6-e367af8aa791","Type":"ContainerStarted","Data":"0f9e72ab84ae40df12cfd74ffc8b464706b72728a5c099ef5f207ce330c60dff"}
Jul 27 15:07:55.824107 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:55.824011    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc" event={"ID":"12742c2c-05be-4268-a2f6-e367af8aa791","Type":"ContainerStarted","Data":"3c2595815be6fe7347cc99d96f189f83700e54f57dce3891489a506a8fa2e961"}
Jul 27 15:07:55.824107 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:55.824035    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc" event={"ID":"12742c2c-05be-4268-a2f6-e367af8aa791","Type":"ContainerStarted","Data":"63549e3bc1d519e07967ec9819355175f39205b147c6fcd0585f1104ba2be61b"}
Jul 27 15:07:55.852162 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:55.852112    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.8323429730000003 podStartE2EDuration="2.852097164s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.754191ms" isStatefulPod=false podCreationTimestamp="2026-07-27 15:07:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:07:55.850999153 +0000 UTC m=+147.279674336" watchObservedRunningTime="2026-07-27 15:07:55.852097164 +0000 UTC m=+147.280772346"
Jul 27 15:07:55.871138 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:55.871097    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-85b764cd7f-8cfkc" podStartSLOduration=1.817183067 podStartE2EDuration="3.871083978s" totalImagesPullingTime="2.053900911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:07:55.870178619 +0000 UTC m=+147.298853828" watchObservedRunningTime="2026-07-27 15:07:55.871083978 +0000 UTC m=+147.299759161"
Jul 27 15:07:56.952171 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:56.952133    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76796964f9-9qlmf"]
Jul 27 15:07:56.954843 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:56.954820    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:56.979556 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:56.968878    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76796964f9-9qlmf"]
Jul 27 15:07:57.013398 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.013343    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:07:57.114227 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.114144    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:07:57.130923 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.130899    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-service-ca\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.131075 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.130941    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-oauth-serving-cert\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.131075 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.130996    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-console-config\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.131075 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.131035    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc8e6d11-c264-4334-97e4-71321167e27f-console-oauth-config\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.131183 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.131091    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc8e6d11-c264-4334-97e4-71321167e27f-console-serving-cert\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.131183 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.131120    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-trusted-ca-bundle\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.131183 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.131135    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brp6k\" (UniqueName: \"kubernetes.io/projected/fc8e6d11-c264-4334-97e4-71321167e27f-kube-api-access-brp6k\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.215224 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.215131    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:07:57.232191 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.232169    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-console-config\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.232257 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.232198    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc8e6d11-c264-4334-97e4-71321167e27f-console-oauth-config\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.232257 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.232234    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc8e6d11-c264-4334-97e4-71321167e27f-console-serving-cert\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.232381 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.232361    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-trusted-ca-bundle\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.232435 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.232401    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brp6k\" (UniqueName: \"kubernetes.io/projected/fc8e6d11-c264-4334-97e4-71321167e27f-kube-api-access-brp6k\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.232486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.232473    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-service-ca\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.232559 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.232514    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-oauth-serving-cert\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.233051 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.232985    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-console-config\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.233286 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.233113    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-service-ca\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.233286 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.233114    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-oauth-serving-cert\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.233431 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.233410    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-trusted-ca-bundle\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.234989 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.234965    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc8e6d11-c264-4334-97e4-71321167e27f-console-oauth-config\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.235354 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.235333    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc8e6d11-c264-4334-97e4-71321167e27f-console-serving-cert\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.241787 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.241770    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brp6k\" (UniqueName: \"kubernetes.io/projected/fc8e6d11-c264-4334-97e4-71321167e27f-kube-api-access-brp6k\") pod \"console-76796964f9-9qlmf\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") " pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.264662 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.264635    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:07:57.393312 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.393280    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76796964f9-9qlmf"]
Jul 27 15:07:57.395924 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:07:57.395900    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc8e6d11_c264_4334_97e4_71321167e27f.slice/crio-32cd295f3e312162486f65889aecc2d8d0625cb31d3217ea4051367bfbc56567 WatchSource:0}: Error finding container 32cd295f3e312162486f65889aecc2d8d0625cb31d3217ea4051367bfbc56567: Status 404 returned error can't find the container with id 32cd295f3e312162486f65889aecc2d8d0625cb31d3217ea4051367bfbc56567
Jul 27 15:07:57.813267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.813207    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:07:57.833235 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.833200    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76796964f9-9qlmf" event={"ID":"fc8e6d11-c264-4334-97e4-71321167e27f","Type":"ContainerStarted","Data":"caecb1c1f75104020ea8351c8e2b1e880b1908507623fb4ceb0188f4ffcdec43"}
Jul 27 15:07:57.833378 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.833258    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76796964f9-9qlmf" event={"ID":"fc8e6d11-c264-4334-97e4-71321167e27f","Type":"ContainerStarted","Data":"32cd295f3e312162486f65889aecc2d8d0625cb31d3217ea4051367bfbc56567"}
Jul 27 15:07:57.854155 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.854104    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76796964f9-9qlmf" podStartSLOduration=1.8540900310000001 podStartE2EDuration="1.854090031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:07:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:07:57.853100701 +0000 UTC m=+149.281775885" watchObservedRunningTime="2026-07-27 15:07:57.854090031 +0000 UTC m=+149.282765214"
Jul 27 15:07:57.908939 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:57.908910    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:07:59.187324 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:07:59.187294    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:08:03.578331 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:03.578296    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:08:07.264868 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:07.264822    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:08:07.265447 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:07.264900    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:08:07.269428 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:07.269404    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:08:07.874668 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:07.874640    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:08:07.921263 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:07.921230    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c594d976d-4vgn5"]
Jul 27 15:08:08.035907 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:08.035877    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:08:08.145087 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:08.145012    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:08:08.238128 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:08.238083    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:08:32.940390 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:32.940347    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c594d976d-4vgn5" podUID="8c882f45-d54e-47ef-956f-9c45950db823" containerName="console" containerID="cri-o://7ae1c6b3e303750be8befa0d7131a61d6ad53118467e3613341f35674c76dd5c" gracePeriod=15
Jul 27 15:08:33.184622 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.184594    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:08:33.345261 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.345227    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c882f45-d54e-47ef-956f-9c45950db823-console-serving-cert\") pod \"8c882f45-d54e-47ef-956f-9c45950db823\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") "
Jul 27 15:08:33.345408 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.345272    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c882f45-d54e-47ef-956f-9c45950db823-console-oauth-config\") pod \"8c882f45-d54e-47ef-956f-9c45950db823\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") "
Jul 27 15:08:33.345408 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.345297    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-trusted-ca-bundle\") pod \"8c882f45-d54e-47ef-956f-9c45950db823\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") "
Jul 27 15:08:33.345408 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.345332    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-oauth-serving-cert\") pod \"8c882f45-d54e-47ef-956f-9c45950db823\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") "
Jul 27 15:08:33.345408 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.345373    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbmqq\" (UniqueName: \"kubernetes.io/projected/8c882f45-d54e-47ef-956f-9c45950db823-kube-api-access-vbmqq\") pod \"8c882f45-d54e-47ef-956f-9c45950db823\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") "
Jul 27 15:08:33.345644 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.345422    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-service-ca\") pod \"8c882f45-d54e-47ef-956f-9c45950db823\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") "
Jul 27 15:08:33.345644 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.345606    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-console-config\") pod \"8c882f45-d54e-47ef-956f-9c45950db823\" (UID: \"8c882f45-d54e-47ef-956f-9c45950db823\") "
Jul 27 15:08:33.345830 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.345806    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-service-ca" (OuterVolumeSpecName: "service-ca") pod "8c882f45-d54e-47ef-956f-9c45950db823" (UID: "8c882f45-d54e-47ef-956f-9c45950db823"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:08:33.346107 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.345815    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8c882f45-d54e-47ef-956f-9c45950db823" (UID: "8c882f45-d54e-47ef-956f-9c45950db823"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:08:33.346107 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.345833    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8c882f45-d54e-47ef-956f-9c45950db823" (UID: "8c882f45-d54e-47ef-956f-9c45950db823"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:08:33.346264 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.346204    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-console-config" (OuterVolumeSpecName: "console-config") pod "8c882f45-d54e-47ef-956f-9c45950db823" (UID: "8c882f45-d54e-47ef-956f-9c45950db823"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:08:33.347757 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.347729    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c882f45-d54e-47ef-956f-9c45950db823-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8c882f45-d54e-47ef-956f-9c45950db823" (UID: "8c882f45-d54e-47ef-956f-9c45950db823"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:08:33.347826 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.347801    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c882f45-d54e-47ef-956f-9c45950db823-kube-api-access-vbmqq" (OuterVolumeSpecName: "kube-api-access-vbmqq") pod "8c882f45-d54e-47ef-956f-9c45950db823" (UID: "8c882f45-d54e-47ef-956f-9c45950db823"). InnerVolumeSpecName "kube-api-access-vbmqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:08:33.347826 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.347807    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c882f45-d54e-47ef-956f-9c45950db823-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8c882f45-d54e-47ef-956f-9c45950db823" (UID: "8c882f45-d54e-47ef-956f-9c45950db823"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:08:33.447210 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.447172    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbmqq\" (UniqueName: \"kubernetes.io/projected/8c882f45-d54e-47ef-956f-9c45950db823-kube-api-access-vbmqq\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:08:33.447210 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.447209    2548 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:08:33.447210 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.447221    2548 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-console-config\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:08:33.447434 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.447228    2548 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c882f45-d54e-47ef-956f-9c45950db823-console-serving-cert\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:08:33.447434 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.447237    2548 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c882f45-d54e-47ef-956f-9c45950db823-console-oauth-config\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:08:33.447434 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.447245    2548 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-trusted-ca-bundle\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:08:33.447434 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.447252    2548 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c882f45-d54e-47ef-956f-9c45950db823-oauth-serving-cert\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:08:33.963452 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.963418    2548 generic.go:356] "Generic (PLEG): container finished" podID="8c882f45-d54e-47ef-956f-9c45950db823" containerID="7ae1c6b3e303750be8befa0d7131a61d6ad53118467e3613341f35674c76dd5c" exitCode=0
Jul 27 15:08:33.963910 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.963463    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c594d976d-4vgn5"
Jul 27 15:08:33.963910 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.963488    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c594d976d-4vgn5" event={"ID":"8c882f45-d54e-47ef-956f-9c45950db823","Type":"ContainerDied","Data":"7ae1c6b3e303750be8befa0d7131a61d6ad53118467e3613341f35674c76dd5c"}
Jul 27 15:08:33.963910 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.963527    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c594d976d-4vgn5" event={"ID":"8c882f45-d54e-47ef-956f-9c45950db823","Type":"ContainerDied","Data":"1e6300c0ab2c9a02b40aa9f4e112cd9426accf6489218f96bdae6529f5a545fa"}
Jul 27 15:08:33.963910 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.963567    2548 scope.go:117] "RemoveContainer" containerID="7ae1c6b3e303750be8befa0d7131a61d6ad53118467e3613341f35674c76dd5c"
Jul 27 15:08:33.972260 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.972244    2548 scope.go:117] "RemoveContainer" containerID="7ae1c6b3e303750be8befa0d7131a61d6ad53118467e3613341f35674c76dd5c"
Jul 27 15:08:33.972497 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:08:33.972481    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ae1c6b3e303750be8befa0d7131a61d6ad53118467e3613341f35674c76dd5c\": container with ID starting with 7ae1c6b3e303750be8befa0d7131a61d6ad53118467e3613341f35674c76dd5c not found: ID does not exist" containerID="7ae1c6b3e303750be8befa0d7131a61d6ad53118467e3613341f35674c76dd5c"
Jul 27 15:08:33.972566 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.972504    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ae1c6b3e303750be8befa0d7131a61d6ad53118467e3613341f35674c76dd5c"} err="failed to get container status \"7ae1c6b3e303750be8befa0d7131a61d6ad53118467e3613341f35674c76dd5c\": rpc error: code = NotFound desc = could not find container \"7ae1c6b3e303750be8befa0d7131a61d6ad53118467e3613341f35674c76dd5c\": container with ID starting with 7ae1c6b3e303750be8befa0d7131a61d6ad53118467e3613341f35674c76dd5c not found: ID does not exist"
Jul 27 15:08:33.984238 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.984216    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c594d976d-4vgn5"]
Jul 27 15:08:33.986067 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:33.986049    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c594d976d-4vgn5"]
Jul 27 15:08:35.138944 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:35.138909    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c882f45-d54e-47ef-956f-9c45950db823" path="/var/lib/kubelet/pods/8c882f45-d54e-47ef-956f-9c45950db823/volumes"
Jul 27 15:08:44.141177 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:44.141137    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:08:44.204238 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:44.204208    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:08:54.186900 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:54.186860    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:08:54.203176 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:54.203150    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:08:55.054618 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:08:55.054589    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 15:09:02.798213 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:02.798156    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:09:02.863704 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:02.863672    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:09:06.554684 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.554650    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-857f974cfb-tbfwc"]
Jul 27 15:09:06.555048 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.554920    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c882f45-d54e-47ef-956f-9c45950db823" containerName="console"
Jul 27 15:09:06.555048 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.554934    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c882f45-d54e-47ef-956f-9c45950db823" containerName="console"
Jul 27 15:09:06.555048 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.554987    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c882f45-d54e-47ef-956f-9c45950db823" containerName="console"
Jul 27 15:09:06.558903 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.558885    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.567905 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.567881    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-857f974cfb-tbfwc"]
Jul 27 15:09:06.605184 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.605147    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:09:06.607875 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.607856    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-config\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.607935 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.607884    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-oauth-config\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.607935 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.607907    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-service-ca\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.607935 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.607929    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-oauth-serving-cert\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.608027 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.607986    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-serving-cert\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.608027 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.608013    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-trusted-ca-bundle\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.608089 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.608046    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmsbb\" (UniqueName: \"kubernetes.io/projected/df5b909a-63e5-456f-bcf2-5b25584f7e08-kube-api-access-qmsbb\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.673960 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.673928    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:09:06.709075 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.709049    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-trusted-ca-bundle\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.709148 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.709102    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmsbb\" (UniqueName: \"kubernetes.io/projected/df5b909a-63e5-456f-bcf2-5b25584f7e08-kube-api-access-qmsbb\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.709183 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.709151    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-config\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.709183 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.709176    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-oauth-config\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.709270 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.709216    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-service-ca\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.709270 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.709235    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-oauth-serving-cert\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.709364 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.709284    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-serving-cert\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.709936 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.709911    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-service-ca\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.710042 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.710012    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-trusted-ca-bundle\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.710094 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.709987    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-config\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.710253 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.710230    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-oauth-serving-cert\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.711888 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.711867    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-oauth-config\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.711983 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.711967    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-serving-cert\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.717781 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.717764    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmsbb\" (UniqueName: \"kubernetes.io/projected/df5b909a-63e5-456f-bcf2-5b25584f7e08-kube-api-access-qmsbb\") pod \"console-857f974cfb-tbfwc\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") " pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.867658 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.867563    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:06.998841 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:06.998817    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-857f974cfb-tbfwc"]
Jul 27 15:09:07.000910 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:09:07.000882    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf5b909a_63e5_456f_bcf2_5b25584f7e08.slice/crio-c218846a945313427007daa63dab1d877351db9fe9a5c17a2f76525f243c5b19 WatchSource:0}: Error finding container c218846a945313427007daa63dab1d877351db9fe9a5c17a2f76525f243c5b19: Status 404 returned error can't find the container with id c218846a945313427007daa63dab1d877351db9fe9a5c17a2f76525f243c5b19
Jul 27 15:09:07.083723 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:07.083688    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857f974cfb-tbfwc" event={"ID":"df5b909a-63e5-456f-bcf2-5b25584f7e08","Type":"ContainerStarted","Data":"4794dff24d528118367d7af9683553aee55512376d83f0f11486bdc4e1c9c042"}
Jul 27 15:09:07.083873 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:07.083744    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857f974cfb-tbfwc" event={"ID":"df5b909a-63e5-456f-bcf2-5b25584f7e08","Type":"ContainerStarted","Data":"c218846a945313427007daa63dab1d877351db9fe9a5c17a2f76525f243c5b19"}
Jul 27 15:09:07.101953 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:07.101906    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-857f974cfb-tbfwc" podStartSLOduration=1.101890282 podStartE2EDuration="1.101890282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:09:07.099988081 +0000 UTC m=+218.528663274" watchObservedRunningTime="2026-07-27 15:09:07.101890282 +0000 UTC m=+218.530565465"
Jul 27 15:09:14.727121 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:14.727083    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:09:14.798710 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:14.798676    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:09:16.867842 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:16.867804    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:16.868339 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:16.867874    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:16.872672 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:16.872649    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:17.121192 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:17.121112    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:09:17.165489 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:17.165459    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76796964f9-9qlmf"]
Jul 27 15:09:17.269195 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:17.269166    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:09:17.369207 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:17.369172    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:09:17.441839 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:17.441764    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:09:32.099811 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:32.099774    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:09:34.344116 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:34.344081    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:09:42.185832 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.185773    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76796964f9-9qlmf" podUID="fc8e6d11-c264-4334-97e4-71321167e27f" containerName="console" containerID="cri-o://caecb1c1f75104020ea8351c8e2b1e880b1908507623fb4ceb0188f4ffcdec43" gracePeriod=15
Jul 27 15:09:42.425253 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.425227    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:09:42.494838 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.494812    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc8e6d11-c264-4334-97e4-71321167e27f-console-serving-cert\") pod \"fc8e6d11-c264-4334-97e4-71321167e27f\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") "
Jul 27 15:09:42.495009 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.494846    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brp6k\" (UniqueName: \"kubernetes.io/projected/fc8e6d11-c264-4334-97e4-71321167e27f-kube-api-access-brp6k\") pod \"fc8e6d11-c264-4334-97e4-71321167e27f\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") "
Jul 27 15:09:42.495009 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.494864    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-oauth-serving-cert\") pod \"fc8e6d11-c264-4334-97e4-71321167e27f\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") "
Jul 27 15:09:42.495009 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.494917    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-trusted-ca-bundle\") pod \"fc8e6d11-c264-4334-97e4-71321167e27f\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") "
Jul 27 15:09:42.495009 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.494936    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-console-config\") pod \"fc8e6d11-c264-4334-97e4-71321167e27f\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") "
Jul 27 15:09:42.495009 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.494963    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-service-ca\") pod \"fc8e6d11-c264-4334-97e4-71321167e27f\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") "
Jul 27 15:09:42.495009 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.494980    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc8e6d11-c264-4334-97e4-71321167e27f-console-oauth-config\") pod \"fc8e6d11-c264-4334-97e4-71321167e27f\" (UID: \"fc8e6d11-c264-4334-97e4-71321167e27f\") "
Jul 27 15:09:42.495422 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.495388    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fc8e6d11-c264-4334-97e4-71321167e27f" (UID: "fc8e6d11-c264-4334-97e4-71321167e27f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:09:42.495489 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.495407    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fc8e6d11-c264-4334-97e4-71321167e27f" (UID: "fc8e6d11-c264-4334-97e4-71321167e27f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:09:42.495489 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.495417    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-console-config" (OuterVolumeSpecName: "console-config") pod "fc8e6d11-c264-4334-97e4-71321167e27f" (UID: "fc8e6d11-c264-4334-97e4-71321167e27f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:09:42.495489 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.495434    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-service-ca" (OuterVolumeSpecName: "service-ca") pod "fc8e6d11-c264-4334-97e4-71321167e27f" (UID: "fc8e6d11-c264-4334-97e4-71321167e27f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:09:42.497158 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.497123    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc8e6d11-c264-4334-97e4-71321167e27f-kube-api-access-brp6k" (OuterVolumeSpecName: "kube-api-access-brp6k") pod "fc8e6d11-c264-4334-97e4-71321167e27f" (UID: "fc8e6d11-c264-4334-97e4-71321167e27f"). InnerVolumeSpecName "kube-api-access-brp6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:09:42.497158 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.497148    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc8e6d11-c264-4334-97e4-71321167e27f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fc8e6d11-c264-4334-97e4-71321167e27f" (UID: "fc8e6d11-c264-4334-97e4-71321167e27f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:09:42.497293 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.497182    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc8e6d11-c264-4334-97e4-71321167e27f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fc8e6d11-c264-4334-97e4-71321167e27f" (UID: "fc8e6d11-c264-4334-97e4-71321167e27f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:09:42.596456 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.596419    2548 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-trusted-ca-bundle\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:09:42.596456 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.596445    2548 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-console-config\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:09:42.596456 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.596455    2548 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:09:42.596456 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.596463    2548 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc8e6d11-c264-4334-97e4-71321167e27f-console-oauth-config\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:09:42.596741 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.596473    2548 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc8e6d11-c264-4334-97e4-71321167e27f-console-serving-cert\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:09:42.596741 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.596481    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brp6k\" (UniqueName: \"kubernetes.io/projected/fc8e6d11-c264-4334-97e4-71321167e27f-kube-api-access-brp6k\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:09:42.596741 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:42.596489    2548 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc8e6d11-c264-4334-97e4-71321167e27f-oauth-serving-cert\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:09:43.209521 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:43.209486    2548 generic.go:356] "Generic (PLEG): container finished" podID="fc8e6d11-c264-4334-97e4-71321167e27f" containerID="caecb1c1f75104020ea8351c8e2b1e880b1908507623fb4ceb0188f4ffcdec43" exitCode=0
Jul 27 15:09:43.209951 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:43.209547    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76796964f9-9qlmf"
Jul 27 15:09:43.209951 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:43.209560    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76796964f9-9qlmf" event={"ID":"fc8e6d11-c264-4334-97e4-71321167e27f","Type":"ContainerDied","Data":"caecb1c1f75104020ea8351c8e2b1e880b1908507623fb4ceb0188f4ffcdec43"}
Jul 27 15:09:43.209951 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:43.209609    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76796964f9-9qlmf" event={"ID":"fc8e6d11-c264-4334-97e4-71321167e27f","Type":"ContainerDied","Data":"32cd295f3e312162486f65889aecc2d8d0625cb31d3217ea4051367bfbc56567"}
Jul 27 15:09:43.209951 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:43.209629    2548 scope.go:117] "RemoveContainer" containerID="caecb1c1f75104020ea8351c8e2b1e880b1908507623fb4ceb0188f4ffcdec43"
Jul 27 15:09:43.217527 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:43.217508    2548 scope.go:117] "RemoveContainer" containerID="caecb1c1f75104020ea8351c8e2b1e880b1908507623fb4ceb0188f4ffcdec43"
Jul 27 15:09:43.217823 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:09:43.217797    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caecb1c1f75104020ea8351c8e2b1e880b1908507623fb4ceb0188f4ffcdec43\": container with ID starting with caecb1c1f75104020ea8351c8e2b1e880b1908507623fb4ceb0188f4ffcdec43 not found: ID does not exist" containerID="caecb1c1f75104020ea8351c8e2b1e880b1908507623fb4ceb0188f4ffcdec43"
Jul 27 15:09:43.217875 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:43.217837    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caecb1c1f75104020ea8351c8e2b1e880b1908507623fb4ceb0188f4ffcdec43"} err="failed to get container status \"caecb1c1f75104020ea8351c8e2b1e880b1908507623fb4ceb0188f4ffcdec43\": rpc error: code = NotFound desc = could not find container \"caecb1c1f75104020ea8351c8e2b1e880b1908507623fb4ceb0188f4ffcdec43\": container with ID starting with caecb1c1f75104020ea8351c8e2b1e880b1908507623fb4ceb0188f4ffcdec43 not found: ID does not exist"
Jul 27 15:09:43.227333 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:43.227311    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76796964f9-9qlmf"]
Jul 27 15:09:43.231484 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:43.231460    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76796964f9-9qlmf"]
Jul 27 15:09:45.137520 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:45.137481    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc8e6d11-c264-4334-97e4-71321167e27f" path="/var/lib/kubelet/pods/fc8e6d11-c264-4334-97e4-71321167e27f/volumes"
Jul 27 15:09:48.734303 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:48.734267    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:09:48.801471 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:48.801436    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:09:59.581201 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:09:59.581160    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:10:29.004141 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:10:29.004109    2548 kubelet.go:1648] "Image garbage collection succeeded"
Jul 27 15:10:38.081619 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:10:38.081580    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:10:38.183618 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:10:38.183584    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:10:59.773410 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:10:59.773370    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:00.675989 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:00.675953    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:03.416774 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:03.416738    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:03.527182 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:03.527152    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:05.232351 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:05.232316    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:05.410099 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:05.410065    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:07.598415 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:07.598377    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:07.971900 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:07.971822    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:08.000710 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.000682    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-t6flj"]
Jul 27 15:11:08.000959 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.000946    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc8e6d11-c264-4334-97e4-71321167e27f" containerName="console"
Jul 27 15:11:08.000959 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.000960    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc8e6d11-c264-4334-97e4-71321167e27f" containerName="console"
Jul 27 15:11:08.001038 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.001013    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc8e6d11-c264-4334-97e4-71321167e27f" containerName="console"
Jul 27 15:11:08.003888 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.003873    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-t6flj"
Jul 27 15:11:08.007082 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.007049    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 27 15:11:08.007196 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.007095    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 27 15:11:08.007961 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.007946    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-4mdh4\""
Jul 27 15:11:08.013881 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.013860    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-t6flj"]
Jul 27 15:11:08.185714 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.185680    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a632054-71bd-441d-9377-d9eb6dce0e81-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-t6flj\" (UID: \"4a632054-71bd-441d-9377-d9eb6dce0e81\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-t6flj"
Jul 27 15:11:08.185714 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.185717    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52vt7\" (UniqueName: \"kubernetes.io/projected/4a632054-71bd-441d-9377-d9eb6dce0e81-kube-api-access-52vt7\") pod \"cert-manager-cainjector-559cdd8c87-t6flj\" (UID: \"4a632054-71bd-441d-9377-d9eb6dce0e81\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-t6flj"
Jul 27 15:11:08.286316 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.286278    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a632054-71bd-441d-9377-d9eb6dce0e81-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-t6flj\" (UID: \"4a632054-71bd-441d-9377-d9eb6dce0e81\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-t6flj"
Jul 27 15:11:08.286454 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.286329    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52vt7\" (UniqueName: \"kubernetes.io/projected/4a632054-71bd-441d-9377-d9eb6dce0e81-kube-api-access-52vt7\") pod \"cert-manager-cainjector-559cdd8c87-t6flj\" (UID: \"4a632054-71bd-441d-9377-d9eb6dce0e81\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-t6flj"
Jul 27 15:11:08.295107 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.295075    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a632054-71bd-441d-9377-d9eb6dce0e81-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-t6flj\" (UID: \"4a632054-71bd-441d-9377-d9eb6dce0e81\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-t6flj"
Jul 27 15:11:08.295277 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.295176    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52vt7\" (UniqueName: \"kubernetes.io/projected/4a632054-71bd-441d-9377-d9eb6dce0e81-kube-api-access-52vt7\") pod \"cert-manager-cainjector-559cdd8c87-t6flj\" (UID: \"4a632054-71bd-441d-9377-d9eb6dce0e81\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-t6flj"
Jul 27 15:11:08.324502 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.324477    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-t6flj"
Jul 27 15:11:08.455608 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.455583    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-t6flj"]
Jul 27 15:11:08.457952 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:11:08.457924    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a632054_71bd_441d_9377_d9eb6dce0e81.slice/crio-1151605abb7618b1135e47af16fffb1ccf5f631f8aa311d6d750cfdb8b4cbf95 WatchSource:0}: Error finding container 1151605abb7618b1135e47af16fffb1ccf5f631f8aa311d6d750cfdb8b4cbf95: Status 404 returned error can't find the container with id 1151605abb7618b1135e47af16fffb1ccf5f631f8aa311d6d750cfdb8b4cbf95
Jul 27 15:11:08.459879 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.459860    2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 15:11:08.513495 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:08.513464    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-t6flj" event={"ID":"4a632054-71bd-441d-9377-d9eb6dce0e81","Type":"ContainerStarted","Data":"1151605abb7618b1135e47af16fffb1ccf5f631f8aa311d6d750cfdb8b4cbf95"}
Jul 27 15:11:10.189518 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:10.189473    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:10.287215 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:10.287174    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:12.531487 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:12.531451    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-t6flj" event={"ID":"4a632054-71bd-441d-9377-d9eb6dce0e81","Type":"ContainerStarted","Data":"0648b430ae349a73d5adf90fcf3be8ea967de2e9a64988d063e6598baa20d016"}
Jul 27 15:11:12.547865 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:12.547813    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-t6flj" podStartSLOduration=1.977896328 podStartE2EDuration="5.547796639s" totalImagesPullingTime="3.569900311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:11:12.547287403 +0000 UTC m=+343.975962586" watchObservedRunningTime="2026-07-27 15:11:12.547796639 +0000 UTC m=+343.976471821"
Jul 27 15:11:12.827644 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:12.827527    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:12.909711 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:12.909681    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:14.770935 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:14.770901    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:21.799170 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:21.799083    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:21.895682 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:21.895651    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:29.378488 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:29.378459    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:29.479376 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:29.479343    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:44.001292 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:44.001250    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:44.104275 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:44.104237    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:44.598015 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:44.597972    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:44.799101 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:44.799063    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:44.905465 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:44.905378    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:44.925917 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:44.925884    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"]
Jul 27 15:11:44.971690 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:44.971649    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"]
Jul 27 15:11:44.971856 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:44.971782    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:44.976339 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:44.976313    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 27 15:11:44.976488 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:44.976402    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 27 15:11:44.978119 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:44.977922    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-srgx6\""
Jul 27 15:11:44.979213 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:44.979192    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 27 15:11:44.980779 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:44.980758    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 27 15:11:45.069159 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.069124    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6516aaf3-43f8-43a7-80c3-72a411ed9497-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.069159 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.069159    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp9fp\" (UniqueName: \"kubernetes.io/projected/6516aaf3-43f8-43a7-80c3-72a411ed9497-kube-api-access-wp9fp\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.069671 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.069188    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6516aaf3-43f8-43a7-80c3-72a411ed9497-manifests\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.069671 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.069326    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6516aaf3-43f8-43a7-80c3-72a411ed9497-tmp\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.069671 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.069378    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6516aaf3-43f8-43a7-80c3-72a411ed9497-webhook-cert\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.113892 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.113860    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:45.169944 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.169849    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6516aaf3-43f8-43a7-80c3-72a411ed9497-tmp\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.169944 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.169898    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6516aaf3-43f8-43a7-80c3-72a411ed9497-webhook-cert\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.169944 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.169938    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6516aaf3-43f8-43a7-80c3-72a411ed9497-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.170185 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.169957    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp9fp\" (UniqueName: \"kubernetes.io/projected/6516aaf3-43f8-43a7-80c3-72a411ed9497-kube-api-access-wp9fp\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.170185 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.169989    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6516aaf3-43f8-43a7-80c3-72a411ed9497-manifests\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.170405 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.170375    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6516aaf3-43f8-43a7-80c3-72a411ed9497-manifests\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.172486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.172458    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6516aaf3-43f8-43a7-80c3-72a411ed9497-tmp\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.172633 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.172616    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6516aaf3-43f8-43a7-80c3-72a411ed9497-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.172728 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.172712    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6516aaf3-43f8-43a7-80c3-72a411ed9497-webhook-cert\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.179634 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.179609    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp9fp\" (UniqueName: \"kubernetes.io/projected/6516aaf3-43f8-43a7-80c3-72a411ed9497-kube-api-access-wp9fp\") pod \"opendatahub-operator-controller-manager-78959f89bb-tr8hk\" (UID: \"6516aaf3-43f8-43a7-80c3-72a411ed9497\") " pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.211924 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.211892    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:45.282526 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.282486    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:45.317422 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.317394    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:45.429755 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.429654    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:45.436289 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.436262    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"]
Jul 27 15:11:45.439699 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:11:45.439665    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6516aaf3_43f8_43a7_80c3_72a411ed9497.slice/crio-e4a3144c39177773901c38b42a4f1cd7b6862a4b554b3741891bbf4506807d10 WatchSource:0}: Error finding container e4a3144c39177773901c38b42a4f1cd7b6862a4b554b3741891bbf4506807d10: Status 404 returned error can't find the container with id e4a3144c39177773901c38b42a4f1cd7b6862a4b554b3741891bbf4506807d10
Jul 27 15:11:45.526510 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.526471    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:45.623922 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.623880    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:45.653098 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:45.653066    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk" event={"ID":"6516aaf3-43f8-43a7-80c3-72a411ed9497","Type":"ContainerStarted","Data":"e4a3144c39177773901c38b42a4f1cd7b6862a4b554b3741891bbf4506807d10"}
Jul 27 15:11:47.642300 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:47.642268    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:47.745889 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:47.745849    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:11:51.682781 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:51.682743    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk" event={"ID":"6516aaf3-43f8-43a7-80c3-72a411ed9497","Type":"ContainerStarted","Data":"6bcd115f965b45220a385105a8047b6281794877dcfae0aed3395069264c661a"}
Jul 27 15:11:52.688453 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:52.688421    2548 generic.go:356] "Generic (PLEG): container finished" podID="6516aaf3-43f8-43a7-80c3-72a411ed9497" containerID="6bcd115f965b45220a385105a8047b6281794877dcfae0aed3395069264c661a" exitCode=0
Jul 27 15:11:52.688876 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:52.688499    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk" event={"ID":"6516aaf3-43f8-43a7-80c3-72a411ed9497","Type":"ContainerDied","Data":"6bcd115f965b45220a385105a8047b6281794877dcfae0aed3395069264c661a"}
Jul 27 15:11:53.693805 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:53.693761    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk" event={"ID":"6516aaf3-43f8-43a7-80c3-72a411ed9497","Type":"ContainerStarted","Data":"872327110eb935fdcdef9153da8ba1da1d966eb853eca6235f67aaf04b313292"}
Jul 27 15:11:53.694184 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:53.693900    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:11:53.717619 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:11:53.717562    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk" podStartSLOduration=2.667349959 podStartE2EDuration="9.717526884s" totalImagesPullingTime="6.010373607s" totalInitContainerRuntime="1.039803318s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:11:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:11:53.715476958 +0000 UTC m=+385.144152140" watchObservedRunningTime="2026-07-27 15:11:53.717526884 +0000 UTC m=+385.146202068"
Jul 27 15:12:04.700239 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:04.700207    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78959f89bb-tr8hk"
Jul 27 15:12:06.624799 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:06.624763    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:06.749067 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:06.749030    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:07.517831 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:07.517805    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:07.621561 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:07.621513    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:07.724666 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:07.724631    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:07.825176 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:07.825076    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:08.174666 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.174575    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:08.206046 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.206015    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wj86m"]
Jul 27 15:12:08.214308 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.214290    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-wj86m"
Jul 27 15:12:08.216889 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.216868    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 27 15:12:08.217021 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.216903    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-jl2wv\""
Jul 27 15:12:08.223734 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.223713    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wj86m"]
Jul 27 15:12:08.360251 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.360215    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlnzw\" (UniqueName: \"kubernetes.io/projected/d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8-kube-api-access-zlnzw\") pod \"model-serving-api-54d89bd5fc-wj86m\" (UID: \"d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8\") " pod="opendatahub/model-serving-api-54d89bd5fc-wj86m"
Jul 27 15:12:08.360436 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.360286    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8-tls-certs\") pod \"model-serving-api-54d89bd5fc-wj86m\" (UID: \"d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8\") " pod="opendatahub/model-serving-api-54d89bd5fc-wj86m"
Jul 27 15:12:08.407186 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.407156    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:08.461130 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.461055    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlnzw\" (UniqueName: \"kubernetes.io/projected/d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8-kube-api-access-zlnzw\") pod \"model-serving-api-54d89bd5fc-wj86m\" (UID: \"d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8\") " pod="opendatahub/model-serving-api-54d89bd5fc-wj86m"
Jul 27 15:12:08.461251 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.461128    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8-tls-certs\") pod \"model-serving-api-54d89bd5fc-wj86m\" (UID: \"d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8\") " pod="opendatahub/model-serving-api-54d89bd5fc-wj86m"
Jul 27 15:12:08.463663 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.463642    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8-tls-certs\") pod \"model-serving-api-54d89bd5fc-wj86m\" (UID: \"d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8\") " pod="opendatahub/model-serving-api-54d89bd5fc-wj86m"
Jul 27 15:12:08.470895 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.470868    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlnzw\" (UniqueName: \"kubernetes.io/projected/d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8-kube-api-access-zlnzw\") pod \"model-serving-api-54d89bd5fc-wj86m\" (UID: \"d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8\") " pod="opendatahub/model-serving-api-54d89bd5fc-wj86m"
Jul 27 15:12:08.524889 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.524859    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-wj86m"
Jul 27 15:12:08.651090 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.650309    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wj86m"]
Jul 27 15:12:08.747823 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:08.747788    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-wj86m" event={"ID":"d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8","Type":"ContainerStarted","Data":"183e59a75e2fe8983269b357f73dd8d4ae67445c6cdd4c61691f06ea47d0a487"}
Jul 27 15:12:11.759614 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:11.759580    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-wj86m" event={"ID":"d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8","Type":"ContainerStarted","Data":"0676dca0224a8b46d2d459ab269ed4c57d483123d5121856accd6644259e332e"}
Jul 27 15:12:11.759987 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:11.759799    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-wj86m"
Jul 27 15:12:11.788017 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:11.787971    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-wj86m" podStartSLOduration=1.333803696 podStartE2EDuration="3.787955937s" totalImagesPullingTime="2.454152241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:12:11.787765288 +0000 UTC m=+403.216440471" watchObservedRunningTime="2026-07-27 15:12:11.787955937 +0000 UTC m=+403.216631120"
Jul 27 15:12:12.535037 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.535005    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:12.669613 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.669584    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns"]
Jul 27 15:12:12.672936 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.672916    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns"
Jul 27 15:12:12.675750 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.675725    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 15:12:12.675871 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.675812    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 15:12:12.675871 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.675799    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 27 15:12:12.675995 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.675787    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 27 15:12:12.676976 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.676961    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-c828q\""
Jul 27 15:12:12.684048 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.684027    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns"]
Jul 27 15:12:12.796998 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.796904    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df921f0f-6513-4002-9a3c-da80931d58a4-tls-certs\") pod \"kube-auth-proxy-cd65dd67b-xsqns\" (UID: \"df921f0f-6513-4002-9a3c-da80931d58a4\") " pod="openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns"
Jul 27 15:12:12.796998 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.796953    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn8zp\" (UniqueName: \"kubernetes.io/projected/df921f0f-6513-4002-9a3c-da80931d58a4-kube-api-access-pn8zp\") pod \"kube-auth-proxy-cd65dd67b-xsqns\" (UID: \"df921f0f-6513-4002-9a3c-da80931d58a4\") " pod="openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns"
Jul 27 15:12:12.796998 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.796986    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df921f0f-6513-4002-9a3c-da80931d58a4-tmp\") pod \"kube-auth-proxy-cd65dd67b-xsqns\" (UID: \"df921f0f-6513-4002-9a3c-da80931d58a4\") " pod="openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns"
Jul 27 15:12:12.829735 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.829704    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:12.897920 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.897882    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df921f0f-6513-4002-9a3c-da80931d58a4-tls-certs\") pod \"kube-auth-proxy-cd65dd67b-xsqns\" (UID: \"df921f0f-6513-4002-9a3c-da80931d58a4\") " pod="openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns"
Jul 27 15:12:12.898091 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.897936    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn8zp\" (UniqueName: \"kubernetes.io/projected/df921f0f-6513-4002-9a3c-da80931d58a4-kube-api-access-pn8zp\") pod \"kube-auth-proxy-cd65dd67b-xsqns\" (UID: \"df921f0f-6513-4002-9a3c-da80931d58a4\") " pod="openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns"
Jul 27 15:12:12.898091 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.897991    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df921f0f-6513-4002-9a3c-da80931d58a4-tmp\") pod \"kube-auth-proxy-cd65dd67b-xsqns\" (UID: \"df921f0f-6513-4002-9a3c-da80931d58a4\") " pod="openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns"
Jul 27 15:12:12.900393 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.900371    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df921f0f-6513-4002-9a3c-da80931d58a4-tmp\") pod \"kube-auth-proxy-cd65dd67b-xsqns\" (UID: \"df921f0f-6513-4002-9a3c-da80931d58a4\") " pod="openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns"
Jul 27 15:12:12.900622 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.900605    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df921f0f-6513-4002-9a3c-da80931d58a4-tls-certs\") pod \"kube-auth-proxy-cd65dd67b-xsqns\" (UID: \"df921f0f-6513-4002-9a3c-da80931d58a4\") " pod="openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns"
Jul 27 15:12:12.908092 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.908063    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn8zp\" (UniqueName: \"kubernetes.io/projected/df921f0f-6513-4002-9a3c-da80931d58a4-kube-api-access-pn8zp\") pod \"kube-auth-proxy-cd65dd67b-xsqns\" (UID: \"df921f0f-6513-4002-9a3c-da80931d58a4\") " pod="openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns"
Jul 27 15:12:12.983409 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:12.983376    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns"
Jul 27 15:12:13.116018 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.115987    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns"]
Jul 27 15:12:13.118814 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:12:13.118788    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf921f0f_6513_4002_9a3c_da80931d58a4.slice/crio-cb049498f0f02d0505b14694fed3c1cdc566fda1491cdc8bf318b2cad899035f WatchSource:0}: Error finding container cb049498f0f02d0505b14694fed3c1cdc566fda1491cdc8bf318b2cad899035f: Status 404 returned error can't find the container with id cb049498f0f02d0505b14694fed3c1cdc566fda1491cdc8bf318b2cad899035f
Jul 27 15:12:13.681129 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.681093    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"]
Jul 27 15:12:13.685407 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.685390    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.688256 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.688222    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 27 15:12:13.688256 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.688245    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 27 15:12:13.688447 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.688324    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 27 15:12:13.688447 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.688343    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bbj94\""
Jul 27 15:12:13.688571 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.688557    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 27 15:12:13.698568 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.698546    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"]
Jul 27 15:12:13.704720 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.704688    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:13.769193 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.769162    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns" event={"ID":"df921f0f-6513-4002-9a3c-da80931d58a4","Type":"ContainerStarted","Data":"cb049498f0f02d0505b14694fed3c1cdc566fda1491cdc8bf318b2cad899035f"}
Jul 27 15:12:13.805610 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.805566    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.806044 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.805653    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/63e24141-c15f-4b37-b2d3-c094c79161f5-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.806044 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.805687    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.806044 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.805728    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.806044 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.805756    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.806044 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.805823    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.806044 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.805903    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7plqf\" (UniqueName: \"kubernetes.io/projected/63e24141-c15f-4b37-b2d3-c094c79161f5-kube-api-access-7plqf\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.809638 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.809612    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:13.907218 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.907173    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.907420 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.907230    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.907420 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.907281    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7plqf\" (UniqueName: \"kubernetes.io/projected/63e24141-c15f-4b37-b2d3-c094c79161f5-kube-api-access-7plqf\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.907420 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.907330    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.907420 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.907383    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/63e24141-c15f-4b37-b2d3-c094c79161f5-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.907420 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.907417    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.907754 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.907452    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.908517 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.908490    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.910289 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.910264    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.910400 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.910369    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.910468 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.910368    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.910605 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.910587    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/63e24141-c15f-4b37-b2d3-c094c79161f5-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.916898 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.916877    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.917389 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.917369    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7plqf\" (UniqueName: \"kubernetes.io/projected/63e24141-c15f-4b37-b2d3-c094c79161f5-kube-api-access-7plqf\") pod \"istiod-openshift-gateway-7b7995d5cd-n8pt4\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:13.995729 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:13.995681    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:14.155738 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:14.155709    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"]
Jul 27 15:12:14.158819 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:12:14.158787    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63e24141_c15f_4b37_b2d3_c094c79161f5.slice/crio-d9149abecf8e599f7ae2c7996a5d056dca356e387aa76a466fc3f399289ec893 WatchSource:0}: Error finding container d9149abecf8e599f7ae2c7996a5d056dca356e387aa76a466fc3f399289ec893: Status 404 returned error can't find the container with id d9149abecf8e599f7ae2c7996a5d056dca356e387aa76a466fc3f399289ec893
Jul 27 15:12:14.775016 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:14.774972    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4" event={"ID":"63e24141-c15f-4b37-b2d3-c094c79161f5","Type":"ContainerStarted","Data":"d9149abecf8e599f7ae2c7996a5d056dca356e387aa76a466fc3f399289ec893"}
Jul 27 15:12:15.391657 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.391618    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"]
Jul 27 15:12:15.396354 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.396320    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.413137 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.413105    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"]
Jul 27 15:12:15.453675 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.453638    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:15.521147 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.521114    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc21e594-c8be-4779-b147-9c0f062cb697-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.521320 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.521155    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc21e594-c8be-4779-b147-9c0f062cb697-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.521320 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.521261    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc21e594-c8be-4779-b147-9c0f062cb697-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.521320 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.521299    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlzr6\" (UniqueName: \"kubernetes.io/projected/dc21e594-c8be-4779-b147-9c0f062cb697-kube-api-access-zlzr6\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.521474 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.521341    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dc21e594-c8be-4779-b147-9c0f062cb697-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.521474 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.521411    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dc21e594-c8be-4779-b147-9c0f062cb697-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.521474 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.521435    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dc21e594-c8be-4779-b147-9c0f062cb697-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.622444 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.622398    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dc21e594-c8be-4779-b147-9c0f062cb697-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.622643 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.622489    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dc21e594-c8be-4779-b147-9c0f062cb697-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.622643 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.622516    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dc21e594-c8be-4779-b147-9c0f062cb697-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.622779 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.622708    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc21e594-c8be-4779-b147-9c0f062cb697-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.622779 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.622757    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc21e594-c8be-4779-b147-9c0f062cb697-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.622889 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.622801    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc21e594-c8be-4779-b147-9c0f062cb697-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.622889 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.622842    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlzr6\" (UniqueName: \"kubernetes.io/projected/dc21e594-c8be-4779-b147-9c0f062cb697-kube-api-access-zlzr6\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.623710 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.623427    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dc21e594-c8be-4779-b147-9c0f062cb697-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.625794 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.625747    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc21e594-c8be-4779-b147-9c0f062cb697-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.625920 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.625808    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dc21e594-c8be-4779-b147-9c0f062cb697-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.625920 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.625827    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dc21e594-c8be-4779-b147-9c0f062cb697-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.626040 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.625978    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc21e594-c8be-4779-b147-9c0f062cb697-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.634694 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.634670    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlzr6\" (UniqueName: \"kubernetes.io/projected/dc21e594-c8be-4779-b147-9c0f062cb697-kube-api-access-zlzr6\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.642819 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.642759    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc21e594-c8be-4779-b147-9c0f062cb697-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fgbbp\" (UID: \"dc21e594-c8be-4779-b147-9c0f062cb697\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.707414 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.707387    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:15.853736 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:15.853688    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"]
Jul 27 15:12:15.857374 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:12:15.857314    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc21e594_c8be_4779_b147_9c0f062cb697.slice/crio-48424f7216fa852da5de086aa78f5a1a53c8ef789bdb0be0b00678a8c68797b8 WatchSource:0}: Error finding container 48424f7216fa852da5de086aa78f5a1a53c8ef789bdb0be0b00678a8c68797b8: Status 404 returned error can't find the container with id 48424f7216fa852da5de086aa78f5a1a53c8ef789bdb0be0b00678a8c68797b8
Jul 27 15:12:16.399390 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:16.399352    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:16.785392 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:16.785351    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns" event={"ID":"df921f0f-6513-4002-9a3c-da80931d58a4","Type":"ContainerStarted","Data":"333d058e2d80966eccf26dc73c473798188d8d8eabb5dce6e586b977871df0ec"}
Jul 27 15:12:16.786693 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:16.786666    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp" event={"ID":"dc21e594-c8be-4779-b147-9c0f062cb697","Type":"ContainerStarted","Data":"48424f7216fa852da5de086aa78f5a1a53c8ef789bdb0be0b00678a8c68797b8"}
Jul 27 15:12:16.810685 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:16.810638    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-cd65dd67b-xsqns" podStartSLOduration=2.173679286 podStartE2EDuration="4.810618139s" totalImagesPullingTime="2.636938853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:12:16.809920597 +0000 UTC m=+408.238595783" watchObservedRunningTime="2026-07-27 15:12:16.810618139 +0000 UTC m=+408.239293324"
Jul 27 15:12:17.579922 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:17.579891    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:17.637071 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:17.637035    2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 15:12:17.637151 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:17.637105    2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 15:12:17.637830 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:17.637588    2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 15:12:17.637830 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:17.637646    2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 15:12:17.700761 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:17.700737    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:17.792589 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:17.792554    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp" event={"ID":"dc21e594-c8be-4779-b147-9c0f062cb697","Type":"ContainerStarted","Data":"69f2e93667346989a2255b128e7a26abe94b9cbd87d649b423ec3f2deb8033c8"}
Jul 27 15:12:17.792742 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:17.792619    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:17.794102 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:17.794074    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4" event={"ID":"63e24141-c15f-4b37-b2d3-c094c79161f5","Type":"ContainerStarted","Data":"510c06787392a732528280470244d73f3f9c865e16a87fd4ec11ddfcee9397ed"}
Jul 27 15:12:17.822902 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:17.822846    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp" podStartSLOduration=1.047240783 podStartE2EDuration="2.822831945s" totalImagesPullingTime="1.775591162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:12:17.820547895 +0000 UTC m=+409.249223074" watchObservedRunningTime="2026-07-27 15:12:17.822831945 +0000 UTC m=+409.251507120"
Jul 27 15:12:17.856081 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:17.855972    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4" podStartSLOduration=1.380073297 podStartE2EDuration="4.855955165s" totalImagesPullingTime="3.475881868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:12:17.85445527 +0000 UTC m=+409.283130454" watchObservedRunningTime="2026-07-27 15:12:17.855955165 +0000 UTC m=+409.284630347"
Jul 27 15:12:17.976736 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:17.976705    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:18.098651 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:18.098611    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:18.198396 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:18.198304    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:18.798693 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:18.798666    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:18.799984 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:18.799952    2548 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-n8pt4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 27 15:12:18.800117 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:18.799993    2548 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-fgbbp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 27 15:12:18.800117 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:18.800003    2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4" podUID="63e24141-c15f-4b37-b2d3-c094c79161f5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 15:12:18.800117 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:18.800033    2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp" podUID="dc21e594-c8be-4779-b147-9c0f062cb697" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 15:12:19.073946 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:19.073857    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:19.186128 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:19.186098    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:19.427358 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:19.427272    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:19.744094 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:19.744063    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:19.802803 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:19.802781    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:19.857557 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:19.857507    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:21.799673 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:21.799647    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fgbbp"
Jul 27 15:12:21.880977 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:21.880948    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"]
Jul 27 15:12:21.881151 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:21.881107    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4" podUID="63e24141-c15f-4b37-b2d3-c094c79161f5" containerName="discovery" containerID="cri-o://510c06787392a732528280470244d73f3f9c865e16a87fd4ec11ddfcee9397ed" gracePeriod=30
Jul 27 15:12:22.147166 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.147143    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:22.180267 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.180243    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-token\") pod \"63e24141-c15f-4b37-b2d3-c094c79161f5\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") "
Jul 27 15:12:22.180433 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.180274    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-cacerts\") pod \"63e24141-c15f-4b37-b2d3-c094c79161f5\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") "
Jul 27 15:12:22.180433 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.180294    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-csr-dns-cert\") pod \"63e24141-c15f-4b37-b2d3-c094c79161f5\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") "
Jul 27 15:12:22.180433 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.180333    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-csr-ca-configmap\") pod \"63e24141-c15f-4b37-b2d3-c094c79161f5\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") "
Jul 27 15:12:22.180433 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.180357    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/63e24141-c15f-4b37-b2d3-c094c79161f5-local-certs\") pod \"63e24141-c15f-4b37-b2d3-c094c79161f5\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") "
Jul 27 15:12:22.180433 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.180390    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-kubeconfig\") pod \"63e24141-c15f-4b37-b2d3-c094c79161f5\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") "
Jul 27 15:12:22.180750 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.180454    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7plqf\" (UniqueName: \"kubernetes.io/projected/63e24141-c15f-4b37-b2d3-c094c79161f5-kube-api-access-7plqf\") pod \"63e24141-c15f-4b37-b2d3-c094c79161f5\" (UID: \"63e24141-c15f-4b37-b2d3-c094c79161f5\") "
Jul 27 15:12:22.180800 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.180763    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "63e24141-c15f-4b37-b2d3-c094c79161f5" (UID: "63e24141-c15f-4b37-b2d3-c094c79161f5"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:12:22.183062 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.183026    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-token" (OuterVolumeSpecName: "istio-token") pod "63e24141-c15f-4b37-b2d3-c094c79161f5" (UID: "63e24141-c15f-4b37-b2d3-c094c79161f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:12:22.183382 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.183345    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "63e24141-c15f-4b37-b2d3-c094c79161f5" (UID: "63e24141-c15f-4b37-b2d3-c094c79161f5"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:12:22.183483 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.183374    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "63e24141-c15f-4b37-b2d3-c094c79161f5" (UID: "63e24141-c15f-4b37-b2d3-c094c79161f5"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:12:22.183483 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.183422    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63e24141-c15f-4b37-b2d3-c094c79161f5-local-certs" (OuterVolumeSpecName: "local-certs") pod "63e24141-c15f-4b37-b2d3-c094c79161f5" (UID: "63e24141-c15f-4b37-b2d3-c094c79161f5"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 15:12:22.183483 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.183451    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-cacerts" (OuterVolumeSpecName: "cacerts") pod "63e24141-c15f-4b37-b2d3-c094c79161f5" (UID: "63e24141-c15f-4b37-b2d3-c094c79161f5"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:12:22.183675 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.183477    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63e24141-c15f-4b37-b2d3-c094c79161f5-kube-api-access-7plqf" (OuterVolumeSpecName: "kube-api-access-7plqf") pod "63e24141-c15f-4b37-b2d3-c094c79161f5" (UID: "63e24141-c15f-4b37-b2d3-c094c79161f5"). InnerVolumeSpecName "kube-api-access-7plqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:12:22.282040 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.282003    2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-token\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:12:22.282040 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.282035    2548 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-cacerts\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:12:22.282040 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.282043    2548 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-csr-dns-cert\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:12:22.282263 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.282054    2548 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-csr-ca-configmap\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:12:22.282263 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.282062    2548 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/63e24141-c15f-4b37-b2d3-c094c79161f5-local-certs\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:12:22.282263 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.282070    2548 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63e24141-c15f-4b37-b2d3-c094c79161f5-istio-kubeconfig\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:12:22.282263 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.282077    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7plqf\" (UniqueName: \"kubernetes.io/projected/63e24141-c15f-4b37-b2d3-c094c79161f5-kube-api-access-7plqf\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:12:22.768883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.768842    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-wj86m"
Jul 27 15:12:22.816786 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.816754    2548 generic.go:356] "Generic (PLEG): container finished" podID="63e24141-c15f-4b37-b2d3-c094c79161f5" containerID="510c06787392a732528280470244d73f3f9c865e16a87fd4ec11ddfcee9397ed" exitCode=0
Jul 27 15:12:22.817253 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.816796    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4" event={"ID":"63e24141-c15f-4b37-b2d3-c094c79161f5","Type":"ContainerDied","Data":"510c06787392a732528280470244d73f3f9c865e16a87fd4ec11ddfcee9397ed"}
Jul 27 15:12:22.817253 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.816833    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"
Jul 27 15:12:22.817253 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.816852    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4" event={"ID":"63e24141-c15f-4b37-b2d3-c094c79161f5","Type":"ContainerDied","Data":"d9149abecf8e599f7ae2c7996a5d056dca356e387aa76a466fc3f399289ec893"}
Jul 27 15:12:22.817253 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.816882    2548 scope.go:117] "RemoveContainer" containerID="510c06787392a732528280470244d73f3f9c865e16a87fd4ec11ddfcee9397ed"
Jul 27 15:12:22.826665 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.826633    2548 scope.go:117] "RemoveContainer" containerID="510c06787392a732528280470244d73f3f9c865e16a87fd4ec11ddfcee9397ed"
Jul 27 15:12:22.826953 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:12:22.826934    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"510c06787392a732528280470244d73f3f9c865e16a87fd4ec11ddfcee9397ed\": container with ID starting with 510c06787392a732528280470244d73f3f9c865e16a87fd4ec11ddfcee9397ed not found: ID does not exist" containerID="510c06787392a732528280470244d73f3f9c865e16a87fd4ec11ddfcee9397ed"
Jul 27 15:12:22.827016 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.826965    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"510c06787392a732528280470244d73f3f9c865e16a87fd4ec11ddfcee9397ed"} err="failed to get container status \"510c06787392a732528280470244d73f3f9c865e16a87fd4ec11ddfcee9397ed\": rpc error: code = NotFound desc = could not find container \"510c06787392a732528280470244d73f3f9c865e16a87fd4ec11ddfcee9397ed\": container with ID starting with 510c06787392a732528280470244d73f3f9c865e16a87fd4ec11ddfcee9397ed not found: ID does not exist"
Jul 27 15:12:22.840898 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.840870    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"]
Jul 27 15:12:22.846022 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:22.845995    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n8pt4"]
Jul 27 15:12:23.138160 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:23.138072    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63e24141-c15f-4b37-b2d3-c094c79161f5" path="/var/lib/kubelet/pods/63e24141-c15f-4b37-b2d3-c094c79161f5/volumes"
Jul 27 15:12:23.445639 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:23.445551    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:31.424149 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:31.424116    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:12:31.525908 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:12:31.525880    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:04.304874 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:04.304783    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:04.363808 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:04.363775    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:05.697169 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:05.697130    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:07.457194 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.457160    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64dc66b447-sr8n8"]
Jul 27 15:13:07.457593 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.457449    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63e24141-c15f-4b37-b2d3-c094c79161f5" containerName="discovery"
Jul 27 15:13:07.457593 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.457459    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="63e24141-c15f-4b37-b2d3-c094c79161f5" containerName="discovery"
Jul 27 15:13:07.457593 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.457516    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="63e24141-c15f-4b37-b2d3-c094c79161f5" containerName="discovery"
Jul 27 15:13:07.461428 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.461408    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.472490 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.472457    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-trusted-ca-bundle\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.472490 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.472488    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-console-config\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.472694 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.472508    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-service-ca\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.472694 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.472524    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-console-serving-cert\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.472694 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.472651    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-oauth-serving-cert\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.472694 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.472686    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-console-oauth-config\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.472936 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.472730    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dq5l\" (UniqueName: \"kubernetes.io/projected/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-kube-api-access-7dq5l\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.473025 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.473007    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64dc66b447-sr8n8"]
Jul 27 15:13:07.520571 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.520528    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:07.573922 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.573883    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-trusted-ca-bundle\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.573922 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.573924    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-console-config\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.574163 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.573944    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-service-ca\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.574163 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.573968    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-console-serving-cert\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.574163 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.574020    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-oauth-serving-cert\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.574163 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.574148    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-console-oauth-config\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.574427 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.574192    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dq5l\" (UniqueName: \"kubernetes.io/projected/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-kube-api-access-7dq5l\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.574750 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.574728    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-service-ca\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.574884 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.574865    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-oauth-serving-cert\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.575084 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.575061    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-console-config\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.575325 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.575309    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-trusted-ca-bundle\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.576709 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.576677    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-console-serving-cert\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.576794 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.576732    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-console-oauth-config\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.584810 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.584790    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dq5l\" (UniqueName: \"kubernetes.io/projected/4171643d-b939-4b6f-8d90-dc3b18f7f9c3-kube-api-access-7dq5l\") pod \"console-64dc66b447-sr8n8\" (UID: \"4171643d-b939-4b6f-8d90-dc3b18f7f9c3\") " pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.624374 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.624343    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:07.770394 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.770360    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:07.896661 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:07.896585    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64dc66b447-sr8n8"]
Jul 27 15:13:07.898821 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:13:07.898791    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4171643d_b939_4b6f_8d90_dc3b18f7f9c3.slice/crio-a958a6b097c8239cf1cb835ba8d63cbcd4642a478ac8306812d9bfd71b3ff627 WatchSource:0}: Error finding container a958a6b097c8239cf1cb835ba8d63cbcd4642a478ac8306812d9bfd71b3ff627: Status 404 returned error can't find the container with id a958a6b097c8239cf1cb835ba8d63cbcd4642a478ac8306812d9bfd71b3ff627
Jul 27 15:13:08.005552 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.005503    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64dc66b447-sr8n8" event={"ID":"4171643d-b939-4b6f-8d90-dc3b18f7f9c3","Type":"ContainerStarted","Data":"f33e2f4bebc82f815637c1c15e897c50c2d3d111d3bf5bf0a95cb10f0732ffc7"}
Jul 27 15:13:08.005693 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.005575    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64dc66b447-sr8n8" event={"ID":"4171643d-b939-4b6f-8d90-dc3b18f7f9c3","Type":"ContainerStarted","Data":"a958a6b097c8239cf1cb835ba8d63cbcd4642a478ac8306812d9bfd71b3ff627"}
Jul 27 15:13:08.024476 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.024371    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64dc66b447-sr8n8" podStartSLOduration=1.024354857 podStartE2EDuration="1.024354857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:13:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:13:08.023321877 +0000 UTC m=+459.451997074" watchObservedRunningTime="2026-07-27 15:13:08.024354857 +0000 UTC m=+459.453030039"
Jul 27 15:13:08.319340 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.319257    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:08.387879 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.387831    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:08.465722 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.465688    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kkj5v"]
Jul 27 15:13:08.467993 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.467832    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kkj5v"
Jul 27 15:13:08.470967 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.470940    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 27 15:13:08.471123 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.470995    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-rf68d\""
Jul 27 15:13:08.471123 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.471117    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 27 15:13:08.478401 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.478375    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kkj5v"]
Jul 27 15:13:08.483119 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.483099    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22z2h\" (UniqueName: \"kubernetes.io/projected/4af27646-fa6b-4cbc-a19f-61d1c8e22ae0-kube-api-access-22z2h\") pod \"kuadrant-operator-catalog-kkj5v\" (UID: \"4af27646-fa6b-4cbc-a19f-61d1c8e22ae0\") " pod="kuadrant-system/kuadrant-operator-catalog-kkj5v"
Jul 27 15:13:08.511220 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.511185    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:08.583912 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.583844    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22z2h\" (UniqueName: \"kubernetes.io/projected/4af27646-fa6b-4cbc-a19f-61d1c8e22ae0-kube-api-access-22z2h\") pod \"kuadrant-operator-catalog-kkj5v\" (UID: \"4af27646-fa6b-4cbc-a19f-61d1c8e22ae0\") " pod="kuadrant-system/kuadrant-operator-catalog-kkj5v"
Jul 27 15:13:08.592700 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.592677    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22z2h\" (UniqueName: \"kubernetes.io/projected/4af27646-fa6b-4cbc-a19f-61d1c8e22ae0-kube-api-access-22z2h\") pod \"kuadrant-operator-catalog-kkj5v\" (UID: \"4af27646-fa6b-4cbc-a19f-61d1c8e22ae0\") " pod="kuadrant-system/kuadrant-operator-catalog-kkj5v"
Jul 27 15:13:08.778503 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.778463    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kkj5v"
Jul 27 15:13:08.825591 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.825369    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kkj5v"]
Jul 27 15:13:08.905581 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:08.905528    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kkj5v"]
Jul 27 15:13:08.908978 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:13:08.908949    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4af27646_fa6b_4cbc_a19f_61d1c8e22ae0.slice/crio-696f007eb4ece60c62596e4eabb8f6db01fa99a9c8e4224bfc649b142f0b1854 WatchSource:0}: Error finding container 696f007eb4ece60c62596e4eabb8f6db01fa99a9c8e4224bfc649b142f0b1854: Status 404 returned error can't find the container with id 696f007eb4ece60c62596e4eabb8f6db01fa99a9c8e4224bfc649b142f0b1854
Jul 27 15:13:09.010993 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:09.010956    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kkj5v" event={"ID":"4af27646-fa6b-4cbc-a19f-61d1c8e22ae0","Type":"ContainerStarted","Data":"696f007eb4ece60c62596e4eabb8f6db01fa99a9c8e4224bfc649b142f0b1854"}
Jul 27 15:13:09.034959 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:09.034928    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wn4ns"]
Jul 27 15:13:09.037690 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:09.037674    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wn4ns"
Jul 27 15:13:09.046974 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:09.046950    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wn4ns"]
Jul 27 15:13:09.088891 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:09.088849    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppwfk\" (UniqueName: \"kubernetes.io/projected/8bc7fa86-8fc8-4efd-8bf1-5b1fdd669742-kube-api-access-ppwfk\") pod \"kuadrant-operator-catalog-wn4ns\" (UID: \"8bc7fa86-8fc8-4efd-8bf1-5b1fdd669742\") " pod="kuadrant-system/kuadrant-operator-catalog-wn4ns"
Jul 27 15:13:09.189811 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:09.189722    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppwfk\" (UniqueName: \"kubernetes.io/projected/8bc7fa86-8fc8-4efd-8bf1-5b1fdd669742-kube-api-access-ppwfk\") pod \"kuadrant-operator-catalog-wn4ns\" (UID: \"8bc7fa86-8fc8-4efd-8bf1-5b1fdd669742\") " pod="kuadrant-system/kuadrant-operator-catalog-wn4ns"
Jul 27 15:13:09.198493 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:09.198463    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppwfk\" (UniqueName: \"kubernetes.io/projected/8bc7fa86-8fc8-4efd-8bf1-5b1fdd669742-kube-api-access-ppwfk\") pod \"kuadrant-operator-catalog-wn4ns\" (UID: \"8bc7fa86-8fc8-4efd-8bf1-5b1fdd669742\") " pod="kuadrant-system/kuadrant-operator-catalog-wn4ns"
Jul 27 15:13:09.346944 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:09.346908    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wn4ns"
Jul 27 15:13:09.473428 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:09.473266    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wn4ns"]
Jul 27 15:13:09.476094 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:13:09.476066    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bc7fa86_8fc8_4efd_8bf1_5b1fdd669742.slice/crio-5a8d1a7ac18a126555eb702fafd59b11edb75f952fe8381001f47156f9f6edf6 WatchSource:0}: Error finding container 5a8d1a7ac18a126555eb702fafd59b11edb75f952fe8381001f47156f9f6edf6: Status 404 returned error can't find the container with id 5a8d1a7ac18a126555eb702fafd59b11edb75f952fe8381001f47156f9f6edf6
Jul 27 15:13:10.021004 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:10.020965    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wn4ns" event={"ID":"8bc7fa86-8fc8-4efd-8bf1-5b1fdd669742","Type":"ContainerStarted","Data":"5a8d1a7ac18a126555eb702fafd59b11edb75f952fe8381001f47156f9f6edf6"}
Jul 27 15:13:12.031221 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:12.031184    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kkj5v" event={"ID":"4af27646-fa6b-4cbc-a19f-61d1c8e22ae0","Type":"ContainerStarted","Data":"c5179d51c7ff15a24b594a1bbef4e7dcf1d90705a5546faf3ed8362aa780cf95"}
Jul 27 15:13:12.031653 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:12.031244    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-kkj5v" podUID="4af27646-fa6b-4cbc-a19f-61d1c8e22ae0" containerName="registry-server" containerID="cri-o://c5179d51c7ff15a24b594a1bbef4e7dcf1d90705a5546faf3ed8362aa780cf95" gracePeriod=2
Jul 27 15:13:12.045433 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:12.045366    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-kkj5v" podStartSLOduration=1.454351526 podStartE2EDuration="4.045347128s" totalImagesPullingTime="2.590995602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:13:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:13:12.04469948 +0000 UTC m=+463.473374701" watchObservedRunningTime="2026-07-27 15:13:12.045347128 +0000 UTC m=+463.474022311"
Jul 27 15:13:12.270895 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:12.270867    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kkj5v"
Jul 27 15:13:12.320525 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:12.320450    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22z2h\" (UniqueName: \"kubernetes.io/projected/4af27646-fa6b-4cbc-a19f-61d1c8e22ae0-kube-api-access-22z2h\") pod \"4af27646-fa6b-4cbc-a19f-61d1c8e22ae0\" (UID: \"4af27646-fa6b-4cbc-a19f-61d1c8e22ae0\") "
Jul 27 15:13:12.322690 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:12.322662    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4af27646-fa6b-4cbc-a19f-61d1c8e22ae0-kube-api-access-22z2h" (OuterVolumeSpecName: "kube-api-access-22z2h") pod "4af27646-fa6b-4cbc-a19f-61d1c8e22ae0" (UID: "4af27646-fa6b-4cbc-a19f-61d1c8e22ae0"). InnerVolumeSpecName "kube-api-access-22z2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:13:12.421745 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:12.421712    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22z2h\" (UniqueName: \"kubernetes.io/projected/4af27646-fa6b-4cbc-a19f-61d1c8e22ae0-kube-api-access-22z2h\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:13:13.036244 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:13.036210    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wn4ns" event={"ID":"8bc7fa86-8fc8-4efd-8bf1-5b1fdd669742","Type":"ContainerStarted","Data":"a8f6fab7118d674ce2b5e15e3d93e3fda388d7224dc238f2feb28bc0209d4445"}
Jul 27 15:13:13.037489 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:13.037465    2548 generic.go:356] "Generic (PLEG): container finished" podID="4af27646-fa6b-4cbc-a19f-61d1c8e22ae0" containerID="c5179d51c7ff15a24b594a1bbef4e7dcf1d90705a5546faf3ed8362aa780cf95" exitCode=0
Jul 27 15:13:13.037641 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:13.037501    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kkj5v"
Jul 27 15:13:13.037641 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:13.037557    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kkj5v" event={"ID":"4af27646-fa6b-4cbc-a19f-61d1c8e22ae0","Type":"ContainerDied","Data":"c5179d51c7ff15a24b594a1bbef4e7dcf1d90705a5546faf3ed8362aa780cf95"}
Jul 27 15:13:13.037641 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:13.037600    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kkj5v" event={"ID":"4af27646-fa6b-4cbc-a19f-61d1c8e22ae0","Type":"ContainerDied","Data":"696f007eb4ece60c62596e4eabb8f6db01fa99a9c8e4224bfc649b142f0b1854"}
Jul 27 15:13:13.037641 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:13.037626    2548 scope.go:117] "RemoveContainer" containerID="c5179d51c7ff15a24b594a1bbef4e7dcf1d90705a5546faf3ed8362aa780cf95"
Jul 27 15:13:13.046370 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:13.046352    2548 scope.go:117] "RemoveContainer" containerID="c5179d51c7ff15a24b594a1bbef4e7dcf1d90705a5546faf3ed8362aa780cf95"
Jul 27 15:13:13.046638 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:13:13.046616    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5179d51c7ff15a24b594a1bbef4e7dcf1d90705a5546faf3ed8362aa780cf95\": container with ID starting with c5179d51c7ff15a24b594a1bbef4e7dcf1d90705a5546faf3ed8362aa780cf95 not found: ID does not exist" containerID="c5179d51c7ff15a24b594a1bbef4e7dcf1d90705a5546faf3ed8362aa780cf95"
Jul 27 15:13:13.046730 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:13.046644    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5179d51c7ff15a24b594a1bbef4e7dcf1d90705a5546faf3ed8362aa780cf95"} err="failed to get container status \"c5179d51c7ff15a24b594a1bbef4e7dcf1d90705a5546faf3ed8362aa780cf95\": rpc error: code = NotFound desc = could not find container \"c5179d51c7ff15a24b594a1bbef4e7dcf1d90705a5546faf3ed8362aa780cf95\": container with ID starting with c5179d51c7ff15a24b594a1bbef4e7dcf1d90705a5546faf3ed8362aa780cf95 not found: ID does not exist"
Jul 27 15:13:13.053503 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:13.053466    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-wn4ns" podStartSLOduration=0.842583988 podStartE2EDuration="4.053454813s" totalImagesPullingTime="3.210870825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:13:13.051101984 +0000 UTC m=+464.479777167" watchObservedRunningTime="2026-07-27 15:13:13.053454813 +0000 UTC m=+464.482129996"
Jul 27 15:13:13.063193 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:13.063161    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kkj5v"]
Jul 27 15:13:13.065178 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:13.065156    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kkj5v"]
Jul 27 15:13:13.138428 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:13.138403    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4af27646-fa6b-4cbc-a19f-61d1c8e22ae0" path="/var/lib/kubelet/pods/4af27646-fa6b-4cbc-a19f-61d1c8e22ae0/volumes"
Jul 27 15:13:13.741146 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:13.741111    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:13.800643 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:13.800586    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:17.770579 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:17.770524    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:17.770974 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:17.770601    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:17.775261 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:17.775237    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:18.062366 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:18.062292    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64dc66b447-sr8n8"
Jul 27 15:13:18.108260 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:18.108228    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-857f974cfb-tbfwc"]
Jul 27 15:13:18.219381 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:18.219351    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:18.322042 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:18.321966    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:18.379226 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:18.379201    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:19.347137 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:19.347103    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-wn4ns"
Jul 27 15:13:19.347595 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:19.347164    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-wn4ns"
Jul 27 15:13:19.368134 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:19.368110    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-wn4ns"
Jul 27 15:13:20.086131 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:20.086106    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-wn4ns"
Jul 27 15:13:37.351879 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:37.351849    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:37.963901 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:37.963865    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:41.414713 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:41.414678    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bdrwf"]
Jul 27 15:13:41.415145 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:41.414977    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4af27646-fa6b-4cbc-a19f-61d1c8e22ae0" containerName="registry-server"
Jul 27 15:13:41.415145 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:41.414988    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="4af27646-fa6b-4cbc-a19f-61d1c8e22ae0" containerName="registry-server"
Jul 27 15:13:41.415145 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:41.415050    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="4af27646-fa6b-4cbc-a19f-61d1c8e22ae0" containerName="registry-server"
Jul 27 15:13:41.418965 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:41.418950    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bdrwf"
Jul 27 15:13:41.422027 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:41.422009    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-xn2jd\""
Jul 27 15:13:41.427782 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:41.427760    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bdrwf"]
Jul 27 15:13:41.569004 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:41.568826    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j29q2\" (UniqueName: \"kubernetes.io/projected/35c40fff-5ea1-4320-ba36-88b589161de8-kube-api-access-j29q2\") pod \"authorino-operator-75cc8d5cb7-bdrwf\" (UID: \"35c40fff-5ea1-4320-ba36-88b589161de8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bdrwf"
Jul 27 15:13:41.670891 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:41.670802    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j29q2\" (UniqueName: \"kubernetes.io/projected/35c40fff-5ea1-4320-ba36-88b589161de8-kube-api-access-j29q2\") pod \"authorino-operator-75cc8d5cb7-bdrwf\" (UID: \"35c40fff-5ea1-4320-ba36-88b589161de8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bdrwf"
Jul 27 15:13:41.682021 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:41.681991    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j29q2\" (UniqueName: \"kubernetes.io/projected/35c40fff-5ea1-4320-ba36-88b589161de8-kube-api-access-j29q2\") pod \"authorino-operator-75cc8d5cb7-bdrwf\" (UID: \"35c40fff-5ea1-4320-ba36-88b589161de8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bdrwf"
Jul 27 15:13:41.729343 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:41.729308    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bdrwf"
Jul 27 15:13:41.894744 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:41.894720    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bdrwf"]
Jul 27 15:13:41.897200 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:13:41.897170    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c40fff_5ea1_4320_ba36_88b589161de8.slice/crio-37e104b703061ba98c855b2c537538e8d42114220a2cf21326522f4fec52df06 WatchSource:0}: Error finding container 37e104b703061ba98c855b2c537538e8d42114220a2cf21326522f4fec52df06: Status 404 returned error can't find the container with id 37e104b703061ba98c855b2c537538e8d42114220a2cf21326522f4fec52df06
Jul 27 15:13:42.158379 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:42.158343    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bdrwf" event={"ID":"35c40fff-5ea1-4320-ba36-88b589161de8","Type":"ContainerStarted","Data":"37e104b703061ba98c855b2c537538e8d42114220a2cf21326522f4fec52df06"}
Jul 27 15:13:43.127662 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:43.127608    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-857f974cfb-tbfwc" podUID="df5b909a-63e5-456f-bcf2-5b25584f7e08" containerName="console" containerID="cri-o://4794dff24d528118367d7af9683553aee55512376d83f0f11486bdc4e1c9c042" gracePeriod=15
Jul 27 15:13:43.159550 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:43.159501    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:44.167353 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.167321    2548 generic.go:356] "Generic (PLEG): container finished" podID="df5b909a-63e5-456f-bcf2-5b25584f7e08" containerID="4794dff24d528118367d7af9683553aee55512376d83f0f11486bdc4e1c9c042" exitCode=0
Jul 27 15:13:44.167727 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.167394    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857f974cfb-tbfwc" event={"ID":"df5b909a-63e5-456f-bcf2-5b25584f7e08","Type":"ContainerDied","Data":"4794dff24d528118367d7af9683553aee55512376d83f0f11486bdc4e1c9c042"}
Jul 27 15:13:44.167727 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.167450    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857f974cfb-tbfwc" event={"ID":"df5b909a-63e5-456f-bcf2-5b25584f7e08","Type":"ContainerDied","Data":"c218846a945313427007daa63dab1d877351db9fe9a5c17a2f76525f243c5b19"}
Jul 27 15:13:44.167727 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.167475    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c218846a945313427007daa63dab1d877351db9fe9a5c17a2f76525f243c5b19"
Jul 27 15:13:44.167727 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.167682    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:13:44.295265 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.295241    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-config\") pod \"df5b909a-63e5-456f-bcf2-5b25584f7e08\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") "
Jul 27 15:13:44.295403 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.295270    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-trusted-ca-bundle\") pod \"df5b909a-63e5-456f-bcf2-5b25584f7e08\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") "
Jul 27 15:13:44.295403 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.295298    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-oauth-serving-cert\") pod \"df5b909a-63e5-456f-bcf2-5b25584f7e08\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") "
Jul 27 15:13:44.295403 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.295332    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-service-ca\") pod \"df5b909a-63e5-456f-bcf2-5b25584f7e08\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") "
Jul 27 15:13:44.295403 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.295352    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-serving-cert\") pod \"df5b909a-63e5-456f-bcf2-5b25584f7e08\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") "
Jul 27 15:13:44.295630 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.295420    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmsbb\" (UniqueName: \"kubernetes.io/projected/df5b909a-63e5-456f-bcf2-5b25584f7e08-kube-api-access-qmsbb\") pod \"df5b909a-63e5-456f-bcf2-5b25584f7e08\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") "
Jul 27 15:13:44.295630 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.295463    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-oauth-config\") pod \"df5b909a-63e5-456f-bcf2-5b25584f7e08\" (UID: \"df5b909a-63e5-456f-bcf2-5b25584f7e08\") "
Jul 27 15:13:44.295736 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.295692    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-config" (OuterVolumeSpecName: "console-config") pod "df5b909a-63e5-456f-bcf2-5b25584f7e08" (UID: "df5b909a-63e5-456f-bcf2-5b25584f7e08"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:13:44.295736 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.295695    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "df5b909a-63e5-456f-bcf2-5b25584f7e08" (UID: "df5b909a-63e5-456f-bcf2-5b25584f7e08"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:13:44.295839 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.295753    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "df5b909a-63e5-456f-bcf2-5b25584f7e08" (UID: "df5b909a-63e5-456f-bcf2-5b25584f7e08"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:13:44.296054 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.296023    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-service-ca" (OuterVolumeSpecName: "service-ca") pod "df5b909a-63e5-456f-bcf2-5b25584f7e08" (UID: "df5b909a-63e5-456f-bcf2-5b25584f7e08"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:13:44.297778 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.297737    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "df5b909a-63e5-456f-bcf2-5b25584f7e08" (UID: "df5b909a-63e5-456f-bcf2-5b25584f7e08"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:13:44.297883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.297783    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "df5b909a-63e5-456f-bcf2-5b25584f7e08" (UID: "df5b909a-63e5-456f-bcf2-5b25584f7e08"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:13:44.297883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.297819    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df5b909a-63e5-456f-bcf2-5b25584f7e08-kube-api-access-qmsbb" (OuterVolumeSpecName: "kube-api-access-qmsbb") pod "df5b909a-63e5-456f-bcf2-5b25584f7e08" (UID: "df5b909a-63e5-456f-bcf2-5b25584f7e08"). InnerVolumeSpecName "kube-api-access-qmsbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:13:44.396109 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.396073    2548 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:13:44.396109 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.396103    2548 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-serving-cert\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:13:44.396109 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.396114    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmsbb\" (UniqueName: \"kubernetes.io/projected/df5b909a-63e5-456f-bcf2-5b25584f7e08-kube-api-access-qmsbb\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:13:44.396337 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.396123    2548 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-oauth-config\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:13:44.396337 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.396131    2548 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-console-config\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:13:44.396337 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.396139    2548 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-trusted-ca-bundle\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:13:44.396337 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:44.396147    2548 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df5b909a-63e5-456f-bcf2-5b25584f7e08-oauth-serving-cert\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:13:45.172836 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.172798    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bdrwf" event={"ID":"35c40fff-5ea1-4320-ba36-88b589161de8","Type":"ContainerStarted","Data":"a13964b92d20e485d9467627efb69fb561818d78ddb45cec3d349e8e31ef8550"}
Jul 27 15:13:45.173199 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.172907    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bdrwf"
Jul 27 15:13:45.173199 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.172937    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857f974cfb-tbfwc"
Jul 27 15:13:45.190685 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.190629    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bdrwf" podStartSLOduration=1.885857678 podStartE2EDuration="4.190614856s" totalImagesPullingTime="2.304757178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:13:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:13:45.189635616 +0000 UTC m=+496.618310798" watchObservedRunningTime="2026-07-27 15:13:45.190614856 +0000 UTC m=+496.619290039"
Jul 27 15:13:45.203072 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.203041    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-857f974cfb-tbfwc"]
Jul 27 15:13:45.207127 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.207105    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-857f974cfb-tbfwc"]
Jul 27 15:13:45.693179 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.693142    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"]
Jul 27 15:13:45.693463 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.693451    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df5b909a-63e5-456f-bcf2-5b25584f7e08" containerName="console"
Jul 27 15:13:45.693523 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.693465    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="df5b909a-63e5-456f-bcf2-5b25584f7e08" containerName="console"
Jul 27 15:13:45.693523 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.693519    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="df5b909a-63e5-456f-bcf2-5b25584f7e08" containerName="console"
Jul 27 15:13:45.697423 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.697403    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"
Jul 27 15:13:45.700130 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.700111    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-vjd2r\""
Jul 27 15:13:45.707698 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.707679    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"]
Jul 27 15:13:45.806637 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.806603    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5n9t\" (UniqueName: \"kubernetes.io/projected/76e2d54d-e7e7-4311-a22f-19aae003df30-kube-api-access-z5n9t\") pod \"limitador-operator-controller-manager-859d7b78bd-spx62\" (UID: \"76e2d54d-e7e7-4311-a22f-19aae003df30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"
Jul 27 15:13:45.907612 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.907576    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5n9t\" (UniqueName: \"kubernetes.io/projected/76e2d54d-e7e7-4311-a22f-19aae003df30-kube-api-access-z5n9t\") pod \"limitador-operator-controller-manager-859d7b78bd-spx62\" (UID: \"76e2d54d-e7e7-4311-a22f-19aae003df30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"
Jul 27 15:13:45.919352 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:45.919318    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5n9t\" (UniqueName: \"kubernetes.io/projected/76e2d54d-e7e7-4311-a22f-19aae003df30-kube-api-access-z5n9t\") pod \"limitador-operator-controller-manager-859d7b78bd-spx62\" (UID: \"76e2d54d-e7e7-4311-a22f-19aae003df30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"
Jul 27 15:13:46.008164 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:46.008130    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"
Jul 27 15:13:46.144294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:46.144265    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"]
Jul 27 15:13:46.147476 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:13:46.147450    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76e2d54d_e7e7_4311_a22f_19aae003df30.slice/crio-b8c8b4659d66c16bbb49cdae007dac8aa6e127c84e533ed9e560caefa421ee73 WatchSource:0}: Error finding container b8c8b4659d66c16bbb49cdae007dac8aa6e127c84e533ed9e560caefa421ee73: Status 404 returned error can't find the container with id b8c8b4659d66c16bbb49cdae007dac8aa6e127c84e533ed9e560caefa421ee73
Jul 27 15:13:46.178180 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:46.178149    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62" event={"ID":"76e2d54d-e7e7-4311-a22f-19aae003df30","Type":"ContainerStarted","Data":"b8c8b4659d66c16bbb49cdae007dac8aa6e127c84e533ed9e560caefa421ee73"}
Jul 27 15:13:47.140720 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:47.138775    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df5b909a-63e5-456f-bcf2-5b25584f7e08" path="/var/lib/kubelet/pods/df5b909a-63e5-456f-bcf2-5b25584f7e08/volumes"
Jul 27 15:13:47.170892 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:47.170859    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:49.676589 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:49.676558    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g"]
Jul 27 15:13:49.680652 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:49.680622    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g"
Jul 27 15:13:49.684083 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:49.684059    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 27 15:13:49.684566 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:49.684525    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-zfqwp\""
Jul 27 15:13:49.692510 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:49.692486    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g"]
Jul 27 15:13:49.843215 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:49.843181    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg6rp\" (UniqueName: \"kubernetes.io/projected/ed328f85-503e-4671-83e9-b44f78f30568-kube-api-access-qg6rp\") pod \"dns-operator-controller-manager-864dd78c4c-2lf8g\" (UID: \"ed328f85-503e-4671-83e9-b44f78f30568\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g"
Jul 27 15:13:49.944358 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:49.944273    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg6rp\" (UniqueName: \"kubernetes.io/projected/ed328f85-503e-4671-83e9-b44f78f30568-kube-api-access-qg6rp\") pod \"dns-operator-controller-manager-864dd78c4c-2lf8g\" (UID: \"ed328f85-503e-4671-83e9-b44f78f30568\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g"
Jul 27 15:13:49.953712 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:49.953681    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg6rp\" (UniqueName: \"kubernetes.io/projected/ed328f85-503e-4671-83e9-b44f78f30568-kube-api-access-qg6rp\") pod \"dns-operator-controller-manager-864dd78c4c-2lf8g\" (UID: \"ed328f85-503e-4671-83e9-b44f78f30568\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g"
Jul 27 15:13:49.992880 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:49.992836    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g"
Jul 27 15:13:50.143805 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:50.143698    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g"]
Jul 27 15:13:50.146654 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:13:50.146625    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded328f85_503e_4671_83e9_b44f78f30568.slice/crio-aa54e3203d4374f7346d36c62b1833b63136b72a9a61d9b1001512f8164c1883 WatchSource:0}: Error finding container aa54e3203d4374f7346d36c62b1833b63136b72a9a61d9b1001512f8164c1883: Status 404 returned error can't find the container with id aa54e3203d4374f7346d36c62b1833b63136b72a9a61d9b1001512f8164c1883
Jul 27 15:13:50.195438 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:50.195356    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g" event={"ID":"ed328f85-503e-4671-83e9-b44f78f30568","Type":"ContainerStarted","Data":"aa54e3203d4374f7346d36c62b1833b63136b72a9a61d9b1001512f8164c1883"}
Jul 27 15:13:50.558376 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:50.558335    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:13:51.201214 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:51.201129    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62" event={"ID":"76e2d54d-e7e7-4311-a22f-19aae003df30","Type":"ContainerStarted","Data":"4d35b4d1e89d1d638356b733acf96fbb454d282a27c9f2442999f63255b6dec0"}
Jul 27 15:13:51.201214 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:51.201208    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"
Jul 27 15:13:51.218420 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:51.218373    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62" podStartSLOduration=1.587708065 podStartE2EDuration="6.218356868s" totalImagesPullingTime="4.630648803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:13:51.217181283 +0000 UTC m=+502.645856466" watchObservedRunningTime="2026-07-27 15:13:51.218356868 +0000 UTC m=+502.647032050"
Jul 27 15:13:53.211626 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:53.211590    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g" event={"ID":"ed328f85-503e-4671-83e9-b44f78f30568","Type":"ContainerStarted","Data":"1bd9c35781851ecc6513f9251bee7306375517719c199f03dea30586b5d4c03f"}
Jul 27 15:13:53.212031 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:53.211715    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g"
Jul 27 15:13:53.229258 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:53.229211    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g" podStartSLOduration=1.926573489 podStartE2EDuration="4.229196169s" totalImagesPullingTime="2.30262268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:13:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:13:53.228036375 +0000 UTC m=+504.656711585" watchObservedRunningTime="2026-07-27 15:13:53.229196169 +0000 UTC m=+504.657871352"
Jul 27 15:13:54.568846 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:54.568810    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"]
Jul 27 15:13:54.572284 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:54.572264    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:13:54.576110 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:54.576079    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rqmqs\""
Jul 27 15:13:54.588638 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:54.588618    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"]
Jul 27 15:13:54.688325 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:54.688283    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-478hv\" (UniqueName: \"kubernetes.io/projected/ee8b2768-8533-4c74-943f-02acd5470401-kube-api-access-478hv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9mbbh\" (UID: \"ee8b2768-8533-4c74-943f-02acd5470401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:13:54.688505 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:54.688367    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee8b2768-8533-4c74-943f-02acd5470401-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9mbbh\" (UID: \"ee8b2768-8533-4c74-943f-02acd5470401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:13:54.789089 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:54.789048    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee8b2768-8533-4c74-943f-02acd5470401-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9mbbh\" (UID: \"ee8b2768-8533-4c74-943f-02acd5470401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:13:54.789249 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:54.789111    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-478hv\" (UniqueName: \"kubernetes.io/projected/ee8b2768-8533-4c74-943f-02acd5470401-kube-api-access-478hv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9mbbh\" (UID: \"ee8b2768-8533-4c74-943f-02acd5470401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:13:54.789507 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:54.789487    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee8b2768-8533-4c74-943f-02acd5470401-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9mbbh\" (UID: \"ee8b2768-8533-4c74-943f-02acd5470401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:13:54.798890 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:54.798865    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-478hv\" (UniqueName: \"kubernetes.io/projected/ee8b2768-8533-4c74-943f-02acd5470401-kube-api-access-478hv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9mbbh\" (UID: \"ee8b2768-8533-4c74-943f-02acd5470401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:13:54.882485 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:54.882408    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:13:55.020633 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:55.020608    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"]
Jul 27 15:13:55.023171 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:13:55.023137    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee8b2768_8533_4c74_943f_02acd5470401.slice/crio-199a04e1933b51a7f2e381461f5e61b2460e10483898647b59ae5b07dfb3faa3 WatchSource:0}: Error finding container 199a04e1933b51a7f2e381461f5e61b2460e10483898647b59ae5b07dfb3faa3: Status 404 returned error can't find the container with id 199a04e1933b51a7f2e381461f5e61b2460e10483898647b59ae5b07dfb3faa3
Jul 27 15:13:55.221314 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:55.221225    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh" event={"ID":"ee8b2768-8533-4c74-943f-02acd5470401","Type":"ContainerStarted","Data":"199a04e1933b51a7f2e381461f5e61b2460e10483898647b59ae5b07dfb3faa3"}
Jul 27 15:13:56.180728 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:13:56.180698    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bdrwf"
Jul 27 15:14:02.207821 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:02.207792    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"
Jul 27 15:14:02.255908 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:02.255879    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh" event={"ID":"ee8b2768-8533-4c74-943f-02acd5470401","Type":"ContainerStarted","Data":"d8bd942153db9d26551b805c99072d1c3f0f2361db121ab1622b88e86ce12c65"}
Jul 27 15:14:02.256253 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:02.256009    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:14:02.276393 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:02.276348    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh" podStartSLOduration=1.582819396 podStartE2EDuration="8.276331835s" totalImagesPullingTime="6.693512439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:14:02.275857183 +0000 UTC m=+513.704532365" watchObservedRunningTime="2026-07-27 15:14:02.276331835 +0000 UTC m=+513.705007017"
Jul 27 15:14:04.218391 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:04.218368    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g"
Jul 27 15:14:04.228904 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:04.228877    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:04.755549 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:04.755503    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:06.357622 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:06.357578    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:06.412981 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:06.412943    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:09.393658 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:09.393631    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:09.451095 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:09.451068    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:10.578438 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:10.578401    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:11.607814 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:11.607782    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:13.118439 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:13.118405    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:13.261958 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:13.261927    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:14:14.264587 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.264554    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"]
Jul 27 15:14:14.268115 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.268099    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"
Jul 27 15:14:14.281006 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.280975    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"]
Jul 27 15:14:14.352485 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.352449    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfqkc\" (UniqueName: \"kubernetes.io/projected/93956f60-5d35-4647-89aa-a2b11befdd82-kube-api-access-tfqkc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j7kln\" (UID: \"93956f60-5d35-4647-89aa-a2b11befdd82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"
Jul 27 15:14:14.352694 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.352500    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/93956f60-5d35-4647-89aa-a2b11befdd82-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j7kln\" (UID: \"93956f60-5d35-4647-89aa-a2b11befdd82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"
Jul 27 15:14:14.453717 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.453683    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfqkc\" (UniqueName: \"kubernetes.io/projected/93956f60-5d35-4647-89aa-a2b11befdd82-kube-api-access-tfqkc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j7kln\" (UID: \"93956f60-5d35-4647-89aa-a2b11befdd82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"
Jul 27 15:14:14.453890 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.453735    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/93956f60-5d35-4647-89aa-a2b11befdd82-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j7kln\" (UID: \"93956f60-5d35-4647-89aa-a2b11befdd82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"
Jul 27 15:14:14.454062 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.454047    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/93956f60-5d35-4647-89aa-a2b11befdd82-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j7kln\" (UID: \"93956f60-5d35-4647-89aa-a2b11befdd82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"
Jul 27 15:14:14.466123 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.466097    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfqkc\" (UniqueName: \"kubernetes.io/projected/93956f60-5d35-4647-89aa-a2b11befdd82-kube-api-access-tfqkc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j7kln\" (UID: \"93956f60-5d35-4647-89aa-a2b11befdd82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"
Jul 27 15:14:14.577899 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.577819    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"
Jul 27 15:14:14.717685 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.717654    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"]
Jul 27 15:14:14.721397 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:14:14.721367    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93956f60_5d35_4647_89aa_a2b11befdd82.slice/crio-561a9084a40a9e110484fb8fac55b66a784632c23b59fcf597af26e199554b54 WatchSource:0}: Error finding container 561a9084a40a9e110484fb8fac55b66a784632c23b59fcf597af26e199554b54: Status 404 returned error can't find the container with id 561a9084a40a9e110484fb8fac55b66a784632c23b59fcf597af26e199554b54
Jul 27 15:14:14.971872 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.971783    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"]
Jul 27 15:14:14.976025 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.975997    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"]
Jul 27 15:14:14.985513 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.985485    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"]
Jul 27 15:14:14.985734 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.985703    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh" podUID="ee8b2768-8533-4c74-943f-02acd5470401" containerName="manager" containerID="cri-o://d8bd942153db9d26551b805c99072d1c3f0f2361db121ab1622b88e86ce12c65" gracePeriod=2
Jul 27 15:14:14.999182 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.999139    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"]
Jul 27 15:14:14.999604 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.999582    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93956f60-5d35-4647-89aa-a2b11befdd82" containerName="manager"
Jul 27 15:14:14.999604 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.999602    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="93956f60-5d35-4647-89aa-a2b11befdd82" containerName="manager"
Jul 27 15:14:14.999798 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:14.999665    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="93956f60-5d35-4647-89aa-a2b11befdd82" containerName="manager"
Jul 27 15:14:15.003376 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:15.003340    2548 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ee8b2768-8533-4c74-943f-02acd5470401\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-9mbbh\": pods \"kuadrant-operator-controller-manager-8684c9ff49-9mbbh\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:14:15.011016 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.010990    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"]
Jul 27 15:14:15.011180 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.011121    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"
Jul 27 15:14:15.012846 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.012825    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"]
Jul 27 15:14:15.013016 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.012997    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62" podUID="76e2d54d-e7e7-4311-a22f-19aae003df30" containerName="manager" containerID="cri-o://4d35b4d1e89d1d638356b733acf96fbb454d282a27c9f2442999f63255b6dec0" gracePeriod=2
Jul 27 15:14:15.014757 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:15.014721    2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9mbbh\" is forbidden: User \"system:node:ip-10-0-143-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-46.ec2.internal' and this object" podUID="ee8b2768-8533-4c74-943f-02acd5470401" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:14:15.017379 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.017314    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"]
Jul 27 15:14:15.019472 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.019419    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"]
Jul 27 15:14:15.036519 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.036495    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkg6b"]
Jul 27 15:14:15.036844 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.036831    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76e2d54d-e7e7-4311-a22f-19aae003df30" containerName="manager"
Jul 27 15:14:15.036909 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.036846    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="76e2d54d-e7e7-4311-a22f-19aae003df30" containerName="manager"
Jul 27 15:14:15.036909 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.036863    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee8b2768-8533-4c74-943f-02acd5470401" containerName="manager"
Jul 27 15:14:15.036909 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.036886    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee8b2768-8533-4c74-943f-02acd5470401" containerName="manager"
Jul 27 15:14:15.037072 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.036941    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="76e2d54d-e7e7-4311-a22f-19aae003df30" containerName="manager"
Jul 27 15:14:15.037072 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.036955    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee8b2768-8533-4c74-943f-02acd5470401" containerName="manager"
Jul 27 15:14:15.041248 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.041228    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkg6b"
Jul 27 15:14:15.043144 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:15.043078    2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-spx62\" is forbidden: User \"system:node:ip-10-0-143-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-46.ec2.internal' and this object" podUID="76e2d54d-e7e7-4311-a22f-19aae003df30" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"
Jul 27 15:14:15.045865 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:15.045820    2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9mbbh\" is forbidden: User \"system:node:ip-10-0-143-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-46.ec2.internal' and this object" podUID="ee8b2768-8533-4c74-943f-02acd5470401" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:14:15.048225 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:15.048190    2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-spx62\" is forbidden: User \"system:node:ip-10-0-143-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-46.ec2.internal' and this object" podUID="76e2d54d-e7e7-4311-a22f-19aae003df30" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"
Jul 27 15:14:15.050441 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:15.050390    2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9mbbh\" is forbidden: User \"system:node:ip-10-0-143-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-46.ec2.internal' and this object" podUID="ee8b2768-8533-4c74-943f-02acd5470401" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:14:15.051182 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.051165    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkg6b"]
Jul 27 15:14:15.058866 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.058836    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/979b0f77-5085-4d27-83a2-2f6f90697443-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pmw9j\" (UID: \"979b0f77-5085-4d27-83a2-2f6f90697443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"
Jul 27 15:14:15.058992 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.058969    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mphjz\" (UniqueName: \"kubernetes.io/projected/979b0f77-5085-4d27-83a2-2f6f90697443-kube-api-access-mphjz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pmw9j\" (UID: \"979b0f77-5085-4d27-83a2-2f6f90697443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"
Jul 27 15:14:15.159594 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.159558    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jllq2\" (UniqueName: \"kubernetes.io/projected/714e3d16-6c58-474f-bab1-d1ef2368096c-kube-api-access-jllq2\") pod \"limitador-operator-controller-manager-859d7b78bd-mkg6b\" (UID: \"714e3d16-6c58-474f-bab1-d1ef2368096c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkg6b"
Jul 27 15:14:15.159800 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.159624    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mphjz\" (UniqueName: \"kubernetes.io/projected/979b0f77-5085-4d27-83a2-2f6f90697443-kube-api-access-mphjz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pmw9j\" (UID: \"979b0f77-5085-4d27-83a2-2f6f90697443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"
Jul 27 15:14:15.159800 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.159743    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/979b0f77-5085-4d27-83a2-2f6f90697443-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pmw9j\" (UID: \"979b0f77-5085-4d27-83a2-2f6f90697443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"
Jul 27 15:14:15.160073 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.160057    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/979b0f77-5085-4d27-83a2-2f6f90697443-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pmw9j\" (UID: \"979b0f77-5085-4d27-83a2-2f6f90697443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"
Jul 27 15:14:15.173912 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.173877    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mphjz\" (UniqueName: \"kubernetes.io/projected/979b0f77-5085-4d27-83a2-2f6f90697443-kube-api-access-mphjz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pmw9j\" (UID: \"979b0f77-5085-4d27-83a2-2f6f90697443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"
Jul 27 15:14:15.246476 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.246452    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:14:15.253728 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.253708    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"
Jul 27 15:14:15.261127 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.261107    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jllq2\" (UniqueName: \"kubernetes.io/projected/714e3d16-6c58-474f-bab1-d1ef2368096c-kube-api-access-jllq2\") pod \"limitador-operator-controller-manager-859d7b78bd-mkg6b\" (UID: \"714e3d16-6c58-474f-bab1-d1ef2368096c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkg6b"
Jul 27 15:14:15.270316 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.270296    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jllq2\" (UniqueName: \"kubernetes.io/projected/714e3d16-6c58-474f-bab1-d1ef2368096c-kube-api-access-jllq2\") pod \"limitador-operator-controller-manager-859d7b78bd-mkg6b\" (UID: \"714e3d16-6c58-474f-bab1-d1ef2368096c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkg6b"
Jul 27 15:14:15.316445 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.316411    2548 generic.go:356] "Generic (PLEG): container finished" podID="ee8b2768-8533-4c74-943f-02acd5470401" containerID="d8bd942153db9d26551b805c99072d1c3f0f2361db121ab1622b88e86ce12c65" exitCode=0
Jul 27 15:14:15.316611 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.316462    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9mbbh"
Jul 27 15:14:15.316611 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.316511    2548 scope.go:117] "RemoveContainer" containerID="d8bd942153db9d26551b805c99072d1c3f0f2361db121ab1622b88e86ce12c65"
Jul 27 15:14:15.318367 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:15.318348    2548 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln" container="manager"
Jul 27 15:14:15.319472 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.319449    2548 generic.go:356] "Generic (PLEG): container finished" podID="76e2d54d-e7e7-4311-a22f-19aae003df30" containerID="4d35b4d1e89d1d638356b733acf96fbb454d282a27c9f2442999f63255b6dec0" exitCode=0
Jul 27 15:14:15.319624 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.319486    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-spx62"
Jul 27 15:14:15.325508 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.325493    2548 scope.go:117] "RemoveContainer" containerID="d8bd942153db9d26551b805c99072d1c3f0f2361db121ab1622b88e86ce12c65"
Jul 27 15:14:15.325847 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:15.325826    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8bd942153db9d26551b805c99072d1c3f0f2361db121ab1622b88e86ce12c65\": container with ID starting with d8bd942153db9d26551b805c99072d1c3f0f2361db121ab1622b88e86ce12c65 not found: ID does not exist" containerID="d8bd942153db9d26551b805c99072d1c3f0f2361db121ab1622b88e86ce12c65"
Jul 27 15:14:15.325898 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.325861    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8bd942153db9d26551b805c99072d1c3f0f2361db121ab1622b88e86ce12c65"} err="failed to get container status \"d8bd942153db9d26551b805c99072d1c3f0f2361db121ab1622b88e86ce12c65\": rpc error: code = NotFound desc = could not find container \"d8bd942153db9d26551b805c99072d1c3f0f2361db121ab1622b88e86ce12c65\": container with ID starting with d8bd942153db9d26551b805c99072d1c3f0f2361db121ab1622b88e86ce12c65 not found: ID does not exist"
Jul 27 15:14:15.325898 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.325880    2548 scope.go:117] "RemoveContainer" containerID="4d35b4d1e89d1d638356b733acf96fbb454d282a27c9f2442999f63255b6dec0"
Jul 27 15:14:15.335411 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.335392    2548 scope.go:117] "RemoveContainer" containerID="4d35b4d1e89d1d638356b733acf96fbb454d282a27c9f2442999f63255b6dec0"
Jul 27 15:14:15.335685 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:15.335667    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d35b4d1e89d1d638356b733acf96fbb454d282a27c9f2442999f63255b6dec0\": container with ID starting with 4d35b4d1e89d1d638356b733acf96fbb454d282a27c9f2442999f63255b6dec0 not found: ID does not exist" containerID="4d35b4d1e89d1d638356b733acf96fbb454d282a27c9f2442999f63255b6dec0"
Jul 27 15:14:15.335759 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.335691    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d35b4d1e89d1d638356b733acf96fbb454d282a27c9f2442999f63255b6dec0"} err="failed to get container status \"4d35b4d1e89d1d638356b733acf96fbb454d282a27c9f2442999f63255b6dec0\": rpc error: code = NotFound desc = could not find container \"4d35b4d1e89d1d638356b733acf96fbb454d282a27c9f2442999f63255b6dec0\": container with ID starting with 4d35b4d1e89d1d638356b733acf96fbb454d282a27c9f2442999f63255b6dec0 not found: ID does not exist"
Jul 27 15:14:15.362368 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.362344    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5n9t\" (UniqueName: \"kubernetes.io/projected/76e2d54d-e7e7-4311-a22f-19aae003df30-kube-api-access-z5n9t\") pod \"76e2d54d-e7e7-4311-a22f-19aae003df30\" (UID: \"76e2d54d-e7e7-4311-a22f-19aae003df30\") "
Jul 27 15:14:15.362471 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.362385    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-478hv\" (UniqueName: \"kubernetes.io/projected/ee8b2768-8533-4c74-943f-02acd5470401-kube-api-access-478hv\") pod \"ee8b2768-8533-4c74-943f-02acd5470401\" (UID: \"ee8b2768-8533-4c74-943f-02acd5470401\") "
Jul 27 15:14:15.362471 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.362460    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee8b2768-8533-4c74-943f-02acd5470401-extensions-socket-volume\") pod \"ee8b2768-8533-4c74-943f-02acd5470401\" (UID: \"ee8b2768-8533-4c74-943f-02acd5470401\") "
Jul 27 15:14:15.362952 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.362918    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee8b2768-8533-4c74-943f-02acd5470401-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ee8b2768-8533-4c74-943f-02acd5470401" (UID: "ee8b2768-8533-4c74-943f-02acd5470401"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 15:14:15.364436 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.364417    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee8b2768-8533-4c74-943f-02acd5470401-kube-api-access-478hv" (OuterVolumeSpecName: "kube-api-access-478hv") pod "ee8b2768-8533-4c74-943f-02acd5470401" (UID: "ee8b2768-8533-4c74-943f-02acd5470401"). InnerVolumeSpecName "kube-api-access-478hv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:14:15.364436 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.364425    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76e2d54d-e7e7-4311-a22f-19aae003df30-kube-api-access-z5n9t" (OuterVolumeSpecName: "kube-api-access-z5n9t") pod "76e2d54d-e7e7-4311-a22f-19aae003df30" (UID: "76e2d54d-e7e7-4311-a22f-19aae003df30"). InnerVolumeSpecName "kube-api-access-z5n9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:14:15.417363 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.417325    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"
Jul 27 15:14:15.423446 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.423427    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkg6b"
Jul 27 15:14:15.463756 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.463675    2548 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee8b2768-8533-4c74-943f-02acd5470401-extensions-socket-volume\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:14:15.463756 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.463714    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5n9t\" (UniqueName: \"kubernetes.io/projected/76e2d54d-e7e7-4311-a22f-19aae003df30-kube-api-access-z5n9t\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:14:15.463756 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.463730    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-478hv\" (UniqueName: \"kubernetes.io/projected/ee8b2768-8533-4c74-943f-02acd5470401-kube-api-access-478hv\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:14:15.566559 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.566514    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"]
Jul 27 15:14:15.569167 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:14:15.569138    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod979b0f77_5085_4d27_83a2_2f6f90697443.slice/crio-08287cb256ca04890997c881f558daf76bfe70a02b86d237a39dee19073251a8 WatchSource:0}: Error finding container 08287cb256ca04890997c881f558daf76bfe70a02b86d237a39dee19073251a8: Status 404 returned error can't find the container with id 08287cb256ca04890997c881f558daf76bfe70a02b86d237a39dee19073251a8
Jul 27 15:14:15.585625 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:15.585603    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkg6b"]
Jul 27 15:14:15.586962 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:14:15.586937    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod714e3d16_6c58_474f_bab1_d1ef2368096c.slice/crio-e0862355e589b8cbcd0cd3d2db92f7bf714f2f920ad3620c62360bb47738f0e4 WatchSource:0}: Error finding container e0862355e589b8cbcd0cd3d2db92f7bf714f2f920ad3620c62360bb47738f0e4: Status 404 returned error can't find the container with id e0862355e589b8cbcd0cd3d2db92f7bf714f2f920ad3620c62360bb47738f0e4
Jul 27 15:14:16.325015 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.324972    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j" event={"ID":"979b0f77-5085-4d27-83a2-2f6f90697443","Type":"ContainerStarted","Data":"ec39ae330c0fde7fab5d162374bc4e78f557c4f64238b51658d84fdb87d460ce"}
Jul 27 15:14:16.325493 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.325032    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j" event={"ID":"979b0f77-5085-4d27-83a2-2f6f90697443","Type":"ContainerStarted","Data":"08287cb256ca04890997c881f558daf76bfe70a02b86d237a39dee19073251a8"}
Jul 27 15:14:16.325493 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.325053    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"
Jul 27 15:14:16.327300 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.327275    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln" podUID="93956f60-5d35-4647-89aa-a2b11befdd82" containerName="manager" containerID="cri-o://8962cb1ed3a38aeec12dab0e90443ee9fb5e35ce529da07b76c5bca48c15e87d" gracePeriod=2
Jul 27 15:14:16.329630 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.329607    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkg6b" event={"ID":"714e3d16-6c58-474f-bab1-d1ef2368096c","Type":"ContainerStarted","Data":"7e33592bbe632887ae73cdaf8317ba527be6a26010769dc93d30dc35fa823abc"}
Jul 27 15:14:16.329745 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.329656    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkg6b" event={"ID":"714e3d16-6c58-474f-bab1-d1ef2368096c","Type":"ContainerStarted","Data":"e0862355e589b8cbcd0cd3d2db92f7bf714f2f920ad3620c62360bb47738f0e4"}
Jul 27 15:14:16.329745 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.329680    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkg6b"
Jul 27 15:14:16.346833 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:16.346804    2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-j7kln\" is forbidden: User \"system:node:ip-10-0-143-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-46.ec2.internal' and this object" podUID="93956f60-5d35-4647-89aa-a2b11befdd82" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"
Jul 27 15:14:16.346971 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.346935    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j" podStartSLOduration=2.346921636 podStartE2EDuration="2.346921636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:14:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:14:16.344387135 +0000 UTC m=+527.773062321" watchObservedRunningTime="2026-07-27 15:14:16.346921636 +0000 UTC m=+527.775596818"
Jul 27 15:14:16.372857 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.372819    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkg6b" podStartSLOduration=1.372803706 podStartE2EDuration="1.372803706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:14:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:14:16.37148837 +0000 UTC m=+527.800163554" watchObservedRunningTime="2026-07-27 15:14:16.372803706 +0000 UTC m=+527.801478889"
Jul 27 15:14:16.537448 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.537417    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"
Jul 27 15:14:16.539894 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:16.539869    2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-j7kln\" is forbidden: User \"system:node:ip-10-0-143-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-46.ec2.internal' and this object" podUID="93956f60-5d35-4647-89aa-a2b11befdd82" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"
Jul 27 15:14:16.571782 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.571761    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/93956f60-5d35-4647-89aa-a2b11befdd82-extensions-socket-volume\") pod \"93956f60-5d35-4647-89aa-a2b11befdd82\" (UID: \"93956f60-5d35-4647-89aa-a2b11befdd82\") "
Jul 27 15:14:16.571937 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.571854    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfqkc\" (UniqueName: \"kubernetes.io/projected/93956f60-5d35-4647-89aa-a2b11befdd82-kube-api-access-tfqkc\") pod \"93956f60-5d35-4647-89aa-a2b11befdd82\" (UID: \"93956f60-5d35-4647-89aa-a2b11befdd82\") "
Jul 27 15:14:16.572128 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.572101    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93956f60-5d35-4647-89aa-a2b11befdd82-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "93956f60-5d35-4647-89aa-a2b11befdd82" (UID: "93956f60-5d35-4647-89aa-a2b11befdd82"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 15:14:16.573917 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.573890    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93956f60-5d35-4647-89aa-a2b11befdd82-kube-api-access-tfqkc" (OuterVolumeSpecName: "kube-api-access-tfqkc") pod "93956f60-5d35-4647-89aa-a2b11befdd82" (UID: "93956f60-5d35-4647-89aa-a2b11befdd82"). InnerVolumeSpecName "kube-api-access-tfqkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:14:16.672465 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.672434    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfqkc\" (UniqueName: \"kubernetes.io/projected/93956f60-5d35-4647-89aa-a2b11befdd82-kube-api-access-tfqkc\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:14:16.672465 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:16.672458    2548 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/93956f60-5d35-4647-89aa-a2b11befdd82-extensions-socket-volume\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:14:17.138739 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:17.138706    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76e2d54d-e7e7-4311-a22f-19aae003df30" path="/var/lib/kubelet/pods/76e2d54d-e7e7-4311-a22f-19aae003df30/volumes"
Jul 27 15:14:17.139070 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:17.139057    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93956f60-5d35-4647-89aa-a2b11befdd82" path="/var/lib/kubelet/pods/93956f60-5d35-4647-89aa-a2b11befdd82/volumes"
Jul 27 15:14:17.139359 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:17.139348    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee8b2768-8533-4c74-943f-02acd5470401" path="/var/lib/kubelet/pods/ee8b2768-8533-4c74-943f-02acd5470401/volumes"
Jul 27 15:14:17.335066 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:17.335027    2548 generic.go:356] "Generic (PLEG): container finished" podID="93956f60-5d35-4647-89aa-a2b11befdd82" containerID="8962cb1ed3a38aeec12dab0e90443ee9fb5e35ce529da07b76c5bca48c15e87d" exitCode=0
Jul 27 15:14:17.335563 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:17.335075    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"
Jul 27 15:14:17.335563 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:17.335093    2548 scope.go:117] "RemoveContainer" containerID="8962cb1ed3a38aeec12dab0e90443ee9fb5e35ce529da07b76c5bca48c15e87d"
Jul 27 15:14:17.340400 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:17.340359    2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-j7kln\" is forbidden: User \"system:node:ip-10-0-143-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-46.ec2.internal' and this object" podUID="93956f60-5d35-4647-89aa-a2b11befdd82" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"
Jul 27 15:14:17.344664 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:17.344644    2548 scope.go:117] "RemoveContainer" containerID="8962cb1ed3a38aeec12dab0e90443ee9fb5e35ce529da07b76c5bca48c15e87d"
Jul 27 15:14:17.344937 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:17.344915    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8962cb1ed3a38aeec12dab0e90443ee9fb5e35ce529da07b76c5bca48c15e87d\": container with ID starting with 8962cb1ed3a38aeec12dab0e90443ee9fb5e35ce529da07b76c5bca48c15e87d not found: ID does not exist" containerID="8962cb1ed3a38aeec12dab0e90443ee9fb5e35ce529da07b76c5bca48c15e87d"
Jul 27 15:14:17.345025 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:17.344942    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8962cb1ed3a38aeec12dab0e90443ee9fb5e35ce529da07b76c5bca48c15e87d"} err="failed to get container status \"8962cb1ed3a38aeec12dab0e90443ee9fb5e35ce529da07b76c5bca48c15e87d\": rpc error: code = NotFound desc = could not find container \"8962cb1ed3a38aeec12dab0e90443ee9fb5e35ce529da07b76c5bca48c15e87d\": container with ID starting with 8962cb1ed3a38aeec12dab0e90443ee9fb5e35ce529da07b76c5bca48c15e87d not found: ID does not exist"
Jul 27 15:14:18.565170 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:18.565128    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:18.669330 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:18.669300    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:18.964367 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:18.964292    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:19.067096 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:19.067021    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:19.137977 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:19.137943    2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-j7kln\" is forbidden: User \"system:node:ip-10-0-143-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-46.ec2.internal' and this object" podUID="93956f60-5d35-4647-89aa-a2b11befdd82" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j7kln"
Jul 27 15:14:22.575002 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:22.574971    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:22.781261 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:22.781231    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:25.968120 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:25.968092    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:26.028602 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:26.028502    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:27.337703 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:27.337670    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"
Jul 27 15:14:27.338139 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:27.337745    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkg6b"
Jul 27 15:14:27.736797 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:27.736767    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"]
Jul 27 15:14:27.743088 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:27.743069    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"
Jul 27 15:14:27.756193 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:27.756167    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"]
Jul 27 15:14:27.877260 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:27.877169    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b70a3472-e01f-4166-88dc-d38fb2f4ff0b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q6mbm\" (UID: \"b70a3472-e01f-4166-88dc-d38fb2f4ff0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"
Jul 27 15:14:27.877260 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:27.877225    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk95k\" (UniqueName: \"kubernetes.io/projected/b70a3472-e01f-4166-88dc-d38fb2f4ff0b-kube-api-access-hk95k\") pod \"kuadrant-operator-controller-manager-86878c769f-q6mbm\" (UID: \"b70a3472-e01f-4166-88dc-d38fb2f4ff0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"
Jul 27 15:14:27.977636 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:27.977594    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b70a3472-e01f-4166-88dc-d38fb2f4ff0b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q6mbm\" (UID: \"b70a3472-e01f-4166-88dc-d38fb2f4ff0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"
Jul 27 15:14:27.977791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:27.977661    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk95k\" (UniqueName: \"kubernetes.io/projected/b70a3472-e01f-4166-88dc-d38fb2f4ff0b-kube-api-access-hk95k\") pod \"kuadrant-operator-controller-manager-86878c769f-q6mbm\" (UID: \"b70a3472-e01f-4166-88dc-d38fb2f4ff0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"
Jul 27 15:14:27.977994 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:27.977975    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b70a3472-e01f-4166-88dc-d38fb2f4ff0b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q6mbm\" (UID: \"b70a3472-e01f-4166-88dc-d38fb2f4ff0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"
Jul 27 15:14:27.988791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:27.988764    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk95k\" (UniqueName: \"kubernetes.io/projected/b70a3472-e01f-4166-88dc-d38fb2f4ff0b-kube-api-access-hk95k\") pod \"kuadrant-operator-controller-manager-86878c769f-q6mbm\" (UID: \"b70a3472-e01f-4166-88dc-d38fb2f4ff0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"
Jul 27 15:14:28.053264 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:28.053239    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"
Jul 27 15:14:28.190082 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:28.190049    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"]
Jul 27 15:14:28.193349 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:14:28.193327    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb70a3472_e01f_4166_88dc_d38fb2f4ff0b.slice/crio-520d6a5b052a0455d7bd4e18025a4c5f83640dbd1f0a4bbf182e77abf3fbf63d WatchSource:0}: Error finding container 520d6a5b052a0455d7bd4e18025a4c5f83640dbd1f0a4bbf182e77abf3fbf63d: Status 404 returned error can't find the container with id 520d6a5b052a0455d7bd4e18025a4c5f83640dbd1f0a4bbf182e77abf3fbf63d
Jul 27 15:14:28.390962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:28.390875    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm" event={"ID":"b70a3472-e01f-4166-88dc-d38fb2f4ff0b","Type":"ContainerStarted","Data":"b899851c0301fabec51585ad0100b2a435ae5ed24fe116b010bd5af714a421e8"}
Jul 27 15:14:28.390962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:28.390929    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm" event={"ID":"b70a3472-e01f-4166-88dc-d38fb2f4ff0b","Type":"ContainerStarted","Data":"520d6a5b052a0455d7bd4e18025a4c5f83640dbd1f0a4bbf182e77abf3fbf63d"}
Jul 27 15:14:28.391437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:28.390998    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"
Jul 27 15:14:28.414430 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:28.414262    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm" podStartSLOduration=1.414242073 podStartE2EDuration="1.414242073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:14:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:14:28.412906311 +0000 UTC m=+539.841581498" watchObservedRunningTime="2026-07-27 15:14:28.414242073 +0000 UTC m=+539.842917257"
Jul 27 15:14:29.924835 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:29.924807    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:30.025261 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:30.025232    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:37.751106 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:37.751076    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:39.396982 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:39.396956    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"
Jul 27 15:14:39.445724 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:39.445695    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"]
Jul 27 15:14:39.445947 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:39.445916    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j" podUID="979b0f77-5085-4d27-83a2-2f6f90697443" containerName="manager" containerID="cri-o://ec39ae330c0fde7fab5d162374bc4e78f557c4f64238b51658d84fdb87d460ce" gracePeriod=10
Jul 27 15:14:39.688104 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:39.688083    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"
Jul 27 15:14:39.875711 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:39.875674    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mphjz\" (UniqueName: \"kubernetes.io/projected/979b0f77-5085-4d27-83a2-2f6f90697443-kube-api-access-mphjz\") pod \"979b0f77-5085-4d27-83a2-2f6f90697443\" (UID: \"979b0f77-5085-4d27-83a2-2f6f90697443\") "
Jul 27 15:14:39.875857 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:39.875722    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/979b0f77-5085-4d27-83a2-2f6f90697443-extensions-socket-volume\") pod \"979b0f77-5085-4d27-83a2-2f6f90697443\" (UID: \"979b0f77-5085-4d27-83a2-2f6f90697443\") "
Jul 27 15:14:39.876083 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:39.876062    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979b0f77-5085-4d27-83a2-2f6f90697443-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "979b0f77-5085-4d27-83a2-2f6f90697443" (UID: "979b0f77-5085-4d27-83a2-2f6f90697443"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 15:14:39.877959 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:39.877941    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/979b0f77-5085-4d27-83a2-2f6f90697443-kube-api-access-mphjz" (OuterVolumeSpecName: "kube-api-access-mphjz") pod "979b0f77-5085-4d27-83a2-2f6f90697443" (UID: "979b0f77-5085-4d27-83a2-2f6f90697443"). InnerVolumeSpecName "kube-api-access-mphjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:14:39.976436 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:39.976416    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mphjz\" (UniqueName: \"kubernetes.io/projected/979b0f77-5085-4d27-83a2-2f6f90697443-kube-api-access-mphjz\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:14:39.976568 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:39.976439    2548 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/979b0f77-5085-4d27-83a2-2f6f90697443-extensions-socket-volume\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:14:40.445716 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:40.445623    2548 generic.go:356] "Generic (PLEG): container finished" podID="979b0f77-5085-4d27-83a2-2f6f90697443" containerID="ec39ae330c0fde7fab5d162374bc4e78f557c4f64238b51658d84fdb87d460ce" exitCode=0
Jul 27 15:14:40.445716 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:40.445670    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"
Jul 27 15:14:40.445716 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:40.445707    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j" event={"ID":"979b0f77-5085-4d27-83a2-2f6f90697443","Type":"ContainerDied","Data":"ec39ae330c0fde7fab5d162374bc4e78f557c4f64238b51658d84fdb87d460ce"}
Jul 27 15:14:40.446269 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:40.445757    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j" event={"ID":"979b0f77-5085-4d27-83a2-2f6f90697443","Type":"ContainerDied","Data":"08287cb256ca04890997c881f558daf76bfe70a02b86d237a39dee19073251a8"}
Jul 27 15:14:40.446269 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:40.445782    2548 scope.go:117] "RemoveContainer" containerID="ec39ae330c0fde7fab5d162374bc4e78f557c4f64238b51658d84fdb87d460ce"
Jul 27 15:14:40.454401 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:40.454383    2548 scope.go:117] "RemoveContainer" containerID="ec39ae330c0fde7fab5d162374bc4e78f557c4f64238b51658d84fdb87d460ce"
Jul 27 15:14:40.454684 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:14:40.454664    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec39ae330c0fde7fab5d162374bc4e78f557c4f64238b51658d84fdb87d460ce\": container with ID starting with ec39ae330c0fde7fab5d162374bc4e78f557c4f64238b51658d84fdb87d460ce not found: ID does not exist" containerID="ec39ae330c0fde7fab5d162374bc4e78f557c4f64238b51658d84fdb87d460ce"
Jul 27 15:14:40.454745 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:40.454693    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec39ae330c0fde7fab5d162374bc4e78f557c4f64238b51658d84fdb87d460ce"} err="failed to get container status \"ec39ae330c0fde7fab5d162374bc4e78f557c4f64238b51658d84fdb87d460ce\": rpc error: code = NotFound desc = could not find container \"ec39ae330c0fde7fab5d162374bc4e78f557c4f64238b51658d84fdb87d460ce\": container with ID starting with ec39ae330c0fde7fab5d162374bc4e78f557c4f64238b51658d84fdb87d460ce not found: ID does not exist"
Jul 27 15:14:40.469650 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:40.469626    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"]
Jul 27 15:14:40.475373 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:40.475352    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pmw9j"]
Jul 27 15:14:41.137917 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:41.137885    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="979b0f77-5085-4d27-83a2-2f6f90697443" path="/var/lib/kubelet/pods/979b0f77-5085-4d27-83a2-2f6f90697443/volumes"
Jul 27 15:14:44.622638 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:44.622608    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:48.729814 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:48.729777    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:48.826856 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:48.826822    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:59.710288 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.710252    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"]
Jul 27 15:14:59.710668 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.710596    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="979b0f77-5085-4d27-83a2-2f6f90697443" containerName="manager"
Jul 27 15:14:59.710668 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.710607    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="979b0f77-5085-4d27-83a2-2f6f90697443" containerName="manager"
Jul 27 15:14:59.710668 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.710663    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="979b0f77-5085-4d27-83a2-2f6f90697443" containerName="manager"
Jul 27 15:14:59.713322 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.713306    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"
Jul 27 15:14:59.715636 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.715612    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:59.715902 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.715883    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 27 15:14:59.716012 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.715922    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nxml9\""
Jul 27 15:14:59.722447 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.722424    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"]
Jul 27 15:14:59.733097 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.733074    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5g7s\" (UniqueName: \"kubernetes.io/projected/2ee964f2-1f9f-45d5-8213-ed8e427878c0-kube-api-access-k5g7s\") pod \"limitador-limitador-64c94fd64f-sh2r6\" (UID: \"2ee964f2-1f9f-45d5-8213-ed8e427878c0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"
Jul 27 15:14:59.733275 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.733111    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2ee964f2-1f9f-45d5-8213-ed8e427878c0-config-file\") pod \"limitador-limitador-64c94fd64f-sh2r6\" (UID: \"2ee964f2-1f9f-45d5-8213-ed8e427878c0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"
Jul 27 15:14:59.806891 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.806859    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"]
Jul 27 15:14:59.813507 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.813466    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:14:59.833880 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.833851    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5g7s\" (UniqueName: \"kubernetes.io/projected/2ee964f2-1f9f-45d5-8213-ed8e427878c0-kube-api-access-k5g7s\") pod \"limitador-limitador-64c94fd64f-sh2r6\" (UID: \"2ee964f2-1f9f-45d5-8213-ed8e427878c0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"
Jul 27 15:14:59.834039 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.833889    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2ee964f2-1f9f-45d5-8213-ed8e427878c0-config-file\") pod \"limitador-limitador-64c94fd64f-sh2r6\" (UID: \"2ee964f2-1f9f-45d5-8213-ed8e427878c0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"
Jul 27 15:14:59.834467 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.834442    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2ee964f2-1f9f-45d5-8213-ed8e427878c0-config-file\") pod \"limitador-limitador-64c94fd64f-sh2r6\" (UID: \"2ee964f2-1f9f-45d5-8213-ed8e427878c0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"
Jul 27 15:14:59.842233 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.842213    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5g7s\" (UniqueName: \"kubernetes.io/projected/2ee964f2-1f9f-45d5-8213-ed8e427878c0-kube-api-access-k5g7s\") pod \"limitador-limitador-64c94fd64f-sh2r6\" (UID: \"2ee964f2-1f9f-45d5-8213-ed8e427878c0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"
Jul 27 15:14:59.874578 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:14:59.874547    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:00.024158 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:00.024124    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"
Jul 27 15:15:00.191169 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:00.191134    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"]
Jul 27 15:15:00.194876 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:15:00.194845    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ee964f2_1f9f_45d5_8213_ed8e427878c0.slice/crio-10a85c9434e6baa6f544646167ee3f60619e6fed96fc96b7ca707596c543764e WatchSource:0}: Error finding container 10a85c9434e6baa6f544646167ee3f60619e6fed96fc96b7ca707596c543764e: Status 404 returned error can't find the container with id 10a85c9434e6baa6f544646167ee3f60619e6fed96fc96b7ca707596c543764e
Jul 27 15:15:00.538502 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:00.538463    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6" event={"ID":"2ee964f2-1f9f-45d5-8213-ed8e427878c0","Type":"ContainerStarted","Data":"10a85c9434e6baa6f544646167ee3f60619e6fed96fc96b7ca707596c543764e"}
Jul 27 15:15:00.563444 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:00.563418    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:00.668241 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:00.668206    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:03.556737 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:03.556708    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6" event={"ID":"2ee964f2-1f9f-45d5-8213-ed8e427878c0","Type":"ContainerStarted","Data":"dcf3d3e00ab0ad5e6b71f115b01bac641a6bb2b232954e1eeb499112045cd6ea"}
Jul 27 15:15:03.557142 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:03.556776    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"
Jul 27 15:15:03.572423 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:03.572369    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6" podStartSLOduration=1.278277388 podStartE2EDuration="4.572350738s" totalImagesPullingTime="3.29407335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:14:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:15:03.57204251 +0000 UTC m=+575.000717693" watchObservedRunningTime="2026-07-27 15:15:03.572350738 +0000 UTC m=+575.001025922"
Jul 27 15:15:14.563209 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:14.563170    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"
Jul 27 15:15:15.191947 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:15.191916    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"]
Jul 27 15:15:15.192144 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:15.192115    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6" podUID="2ee964f2-1f9f-45d5-8213-ed8e427878c0" containerName="limitador" containerID="cri-o://dcf3d3e00ab0ad5e6b71f115b01bac641a6bb2b232954e1eeb499112045cd6ea" gracePeriod=30
Jul 27 15:15:15.492428 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:15.492398    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:15.612019 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:15.611985    2548 generic.go:356] "Generic (PLEG): container finished" podID="2ee964f2-1f9f-45d5-8213-ed8e427878c0" containerID="dcf3d3e00ab0ad5e6b71f115b01bac641a6bb2b232954e1eeb499112045cd6ea" exitCode=0
Jul 27 15:15:15.612400 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:15.612027    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6" event={"ID":"2ee964f2-1f9f-45d5-8213-ed8e427878c0","Type":"ContainerDied","Data":"dcf3d3e00ab0ad5e6b71f115b01bac641a6bb2b232954e1eeb499112045cd6ea"}
Jul 27 15:15:16.143571 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:16.143526    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"
Jul 27 15:15:16.290001 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:16.289922    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5g7s\" (UniqueName: \"kubernetes.io/projected/2ee964f2-1f9f-45d5-8213-ed8e427878c0-kube-api-access-k5g7s\") pod \"2ee964f2-1f9f-45d5-8213-ed8e427878c0\" (UID: \"2ee964f2-1f9f-45d5-8213-ed8e427878c0\") "
Jul 27 15:15:16.290001 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:16.289988    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2ee964f2-1f9f-45d5-8213-ed8e427878c0-config-file\") pod \"2ee964f2-1f9f-45d5-8213-ed8e427878c0\" (UID: \"2ee964f2-1f9f-45d5-8213-ed8e427878c0\") "
Jul 27 15:15:16.290368 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:16.290343    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ee964f2-1f9f-45d5-8213-ed8e427878c0-config-file" (OuterVolumeSpecName: "config-file") pod "2ee964f2-1f9f-45d5-8213-ed8e427878c0" (UID: "2ee964f2-1f9f-45d5-8213-ed8e427878c0"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:15:16.292212 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:16.292192    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ee964f2-1f9f-45d5-8213-ed8e427878c0-kube-api-access-k5g7s" (OuterVolumeSpecName: "kube-api-access-k5g7s") pod "2ee964f2-1f9f-45d5-8213-ed8e427878c0" (UID: "2ee964f2-1f9f-45d5-8213-ed8e427878c0"). InnerVolumeSpecName "kube-api-access-k5g7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:15:16.390929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:16.390892    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5g7s\" (UniqueName: \"kubernetes.io/projected/2ee964f2-1f9f-45d5-8213-ed8e427878c0-kube-api-access-k5g7s\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:15:16.390929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:16.390923    2548 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2ee964f2-1f9f-45d5-8213-ed8e427878c0-config-file\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:15:16.617795 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:16.617704    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"
Jul 27 15:15:16.617795 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:16.617733    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sh2r6" event={"ID":"2ee964f2-1f9f-45d5-8213-ed8e427878c0","Type":"ContainerDied","Data":"10a85c9434e6baa6f544646167ee3f60619e6fed96fc96b7ca707596c543764e"}
Jul 27 15:15:16.618270 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:16.617803    2548 scope.go:117] "RemoveContainer" containerID="dcf3d3e00ab0ad5e6b71f115b01bac641a6bb2b232954e1eeb499112045cd6ea"
Jul 27 15:15:16.642860 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:16.642828    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"]
Jul 27 15:15:16.646344 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:16.646317    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sh2r6"]
Jul 27 15:15:17.138107 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:17.138062    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ee964f2-1f9f-45d5-8213-ed8e427878c0" path="/var/lib/kubelet/pods/2ee964f2-1f9f-45d5-8213-ed8e427878c0/volumes"
Jul 27 15:15:17.674595 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:17.674562    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:17.774709 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:17.774672    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:18.148858 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.148827    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv"]
Jul 27 15:15:18.149238 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.149220    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ee964f2-1f9f-45d5-8213-ed8e427878c0" containerName="limitador"
Jul 27 15:15:18.149326 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.149241    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ee964f2-1f9f-45d5-8213-ed8e427878c0" containerName="limitador"
Jul 27 15:15:18.149380 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.149358    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ee964f2-1f9f-45d5-8213-ed8e427878c0" containerName="limitador"
Jul 27 15:15:18.153656 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.153633    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv"
Jul 27 15:15:18.161947 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.161906    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 15:15:18.163064 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.162639    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 27 15:15:18.163064 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.162857    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-5hcd2\""
Jul 27 15:15:18.163064 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.162997    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 27 15:15:18.165692 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.165664    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv"]
Jul 27 15:15:18.180929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.180899    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:18.308675 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.308641    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6ee67b26-8e48-4370-9a20-223b9238e72f-data\") pod \"postgres-69fb4594cf-vz5dv\" (UID: \"6ee67b26-8e48-4370-9a20-223b9238e72f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv"
Jul 27 15:15:18.308852 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.308689    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc42v\" (UniqueName: \"kubernetes.io/projected/6ee67b26-8e48-4370-9a20-223b9238e72f-kube-api-access-wc42v\") pod \"postgres-69fb4594cf-vz5dv\" (UID: \"6ee67b26-8e48-4370-9a20-223b9238e72f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv"
Jul 27 15:15:18.410192 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.410089    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6ee67b26-8e48-4370-9a20-223b9238e72f-data\") pod \"postgres-69fb4594cf-vz5dv\" (UID: \"6ee67b26-8e48-4370-9a20-223b9238e72f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv"
Jul 27 15:15:18.410192 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.410135    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc42v\" (UniqueName: \"kubernetes.io/projected/6ee67b26-8e48-4370-9a20-223b9238e72f-kube-api-access-wc42v\") pod \"postgres-69fb4594cf-vz5dv\" (UID: \"6ee67b26-8e48-4370-9a20-223b9238e72f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv"
Jul 27 15:15:18.410527 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.410504    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6ee67b26-8e48-4370-9a20-223b9238e72f-data\") pod \"postgres-69fb4594cf-vz5dv\" (UID: \"6ee67b26-8e48-4370-9a20-223b9238e72f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv"
Jul 27 15:15:18.418646 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.418618    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc42v\" (UniqueName: \"kubernetes.io/projected/6ee67b26-8e48-4370-9a20-223b9238e72f-kube-api-access-wc42v\") pod \"postgres-69fb4594cf-vz5dv\" (UID: \"6ee67b26-8e48-4370-9a20-223b9238e72f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv"
Jul 27 15:15:18.471771 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.471735    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv"
Jul 27 15:15:18.609236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.609213    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv"]
Jul 27 15:15:18.611910 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:15:18.611881    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ee67b26_8e48_4370_9a20_223b9238e72f.slice/crio-815e9800785af488eff1d4fa42392a1c979e96e69f74d0a4f4e234377c595f48 WatchSource:0}: Error finding container 815e9800785af488eff1d4fa42392a1c979e96e69f74d0a4f4e234377c595f48: Status 404 returned error can't find the container with id 815e9800785af488eff1d4fa42392a1c979e96e69f74d0a4f4e234377c595f48
Jul 27 15:15:18.630150 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:18.630118    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv" event={"ID":"6ee67b26-8e48-4370-9a20-223b9238e72f","Type":"ContainerStarted","Data":"815e9800785af488eff1d4fa42392a1c979e96e69f74d0a4f4e234377c595f48"}
Jul 27 15:15:23.661197 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:23.661158    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv" event={"ID":"6ee67b26-8e48-4370-9a20-223b9238e72f","Type":"ContainerStarted","Data":"85df26a83d2581fdbe8cce3a783520ccd3d7e1c0ea0e8b369184f45563ac9a1b"}
Jul 27 15:15:23.661652 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:23.661281    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv"
Jul 27 15:15:23.679199 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:23.679144    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv" podStartSLOduration=0.800607511 podStartE2EDuration="5.679126242s" totalImagesPullingTime="4.878518731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:15:23.677975104 +0000 UTC m=+595.106650288" watchObservedRunningTime="2026-07-27 15:15:23.679126242 +0000 UTC m=+595.107801425"
Jul 27 15:15:29.062153 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:29.062123    2548 scope.go:117] "RemoveContainer" containerID="4794dff24d528118367d7af9683553aee55512376d83f0f11486bdc4e1c9c042"
Jul 27 15:15:29.695846 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:29.695817    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vz5dv"
Jul 27 15:15:30.476112 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:30.476080    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:30.540847 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:30.540808    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:38.460926 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.460883    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:38.481840 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.481811    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57b4cbfdf4-qcvqn"]
Jul 27 15:15:38.485066 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.485045    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn"
Jul 27 15:15:38.487777 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.487755    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 15:15:38.487902 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.487760    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tbchg\""
Jul 27 15:15:38.487902 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.487797    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 27 15:15:38.495336 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.495312    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b4cbfdf4-qcvqn"]
Jul 27 15:15:38.563876 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.563844    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:38.591843 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.591809    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3483f945-08fc-47e3-8c17-49c3be442ba4-webhook-cert\") pod \"maas-controller-57b4cbfdf4-qcvqn\" (UID: \"3483f945-08fc-47e3-8c17-49c3be442ba4\") " pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn"
Jul 27 15:15:38.591974 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.591864    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqz6s\" (UniqueName: \"kubernetes.io/projected/3483f945-08fc-47e3-8c17-49c3be442ba4-kube-api-access-sqz6s\") pod \"maas-controller-57b4cbfdf4-qcvqn\" (UID: \"3483f945-08fc-47e3-8c17-49c3be442ba4\") " pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn"
Jul 27 15:15:38.615570 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.615520    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:38.662555 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.662513    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-kt6f4"]
Jul 27 15:15:38.666032 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.666011    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cf85d6c8-kt6f4"
Jul 27 15:15:38.675204 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.675183    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-kt6f4"]
Jul 27 15:15:38.692592 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.692569    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3483f945-08fc-47e3-8c17-49c3be442ba4-webhook-cert\") pod \"maas-controller-57b4cbfdf4-qcvqn\" (UID: \"3483f945-08fc-47e3-8c17-49c3be442ba4\") " pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn"
Jul 27 15:15:38.692715 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.692623    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqz6s\" (UniqueName: \"kubernetes.io/projected/3483f945-08fc-47e3-8c17-49c3be442ba4-kube-api-access-sqz6s\") pod \"maas-controller-57b4cbfdf4-qcvqn\" (UID: \"3483f945-08fc-47e3-8c17-49c3be442ba4\") " pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn"
Jul 27 15:15:38.692715 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.692683    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjhk2\" (UniqueName: \"kubernetes.io/projected/11314a3b-d87e-4b19-86ed-034d1b9eb6be-kube-api-access-zjhk2\") pod \"maas-controller-77cf85d6c8-kt6f4\" (UID: \"11314a3b-d87e-4b19-86ed-034d1b9eb6be\") " pod="opendatahub/maas-controller-77cf85d6c8-kt6f4"
Jul 27 15:15:38.692834 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.692716    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11314a3b-d87e-4b19-86ed-034d1b9eb6be-webhook-cert\") pod \"maas-controller-77cf85d6c8-kt6f4\" (UID: \"11314a3b-d87e-4b19-86ed-034d1b9eb6be\") " pod="opendatahub/maas-controller-77cf85d6c8-kt6f4"
Jul 27 15:15:38.695510 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.695483    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3483f945-08fc-47e3-8c17-49c3be442ba4-webhook-cert\") pod \"maas-controller-57b4cbfdf4-qcvqn\" (UID: \"3483f945-08fc-47e3-8c17-49c3be442ba4\") " pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn"
Jul 27 15:15:38.704051 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.704031    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqz6s\" (UniqueName: \"kubernetes.io/projected/3483f945-08fc-47e3-8c17-49c3be442ba4-kube-api-access-sqz6s\") pod \"maas-controller-57b4cbfdf4-qcvqn\" (UID: \"3483f945-08fc-47e3-8c17-49c3be442ba4\") " pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn"
Jul 27 15:15:38.750046 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.750011    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:38.793420 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.793386    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjhk2\" (UniqueName: \"kubernetes.io/projected/11314a3b-d87e-4b19-86ed-034d1b9eb6be-kube-api-access-zjhk2\") pod \"maas-controller-77cf85d6c8-kt6f4\" (UID: \"11314a3b-d87e-4b19-86ed-034d1b9eb6be\") " pod="opendatahub/maas-controller-77cf85d6c8-kt6f4"
Jul 27 15:15:38.793565 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.793426    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11314a3b-d87e-4b19-86ed-034d1b9eb6be-webhook-cert\") pod \"maas-controller-77cf85d6c8-kt6f4\" (UID: \"11314a3b-d87e-4b19-86ed-034d1b9eb6be\") " pod="opendatahub/maas-controller-77cf85d6c8-kt6f4"
Jul 27 15:15:38.795731 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.795713    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn"
Jul 27 15:15:38.796095 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.796074    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11314a3b-d87e-4b19-86ed-034d1b9eb6be-webhook-cert\") pod \"maas-controller-77cf85d6c8-kt6f4\" (UID: \"11314a3b-d87e-4b19-86ed-034d1b9eb6be\") " pod="opendatahub/maas-controller-77cf85d6c8-kt6f4"
Jul 27 15:15:38.802844 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.802823    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjhk2\" (UniqueName: \"kubernetes.io/projected/11314a3b-d87e-4b19-86ed-034d1b9eb6be-kube-api-access-zjhk2\") pod \"maas-controller-77cf85d6c8-kt6f4\" (UID: \"11314a3b-d87e-4b19-86ed-034d1b9eb6be\") " pod="opendatahub/maas-controller-77cf85d6c8-kt6f4"
Jul 27 15:15:38.927626 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.927605    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b4cbfdf4-qcvqn"]
Jul 27 15:15:38.929526 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:15:38.929497    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3483f945_08fc_47e3_8c17_49c3be442ba4.slice/crio-4ff1265fbcf7362c1215bffdfe062f96a49b8e967921c395a9ac0f4c6fb982bc WatchSource:0}: Error finding container 4ff1265fbcf7362c1215bffdfe062f96a49b8e967921c395a9ac0f4c6fb982bc: Status 404 returned error can't find the container with id 4ff1265fbcf7362c1215bffdfe062f96a49b8e967921c395a9ac0f4c6fb982bc
Jul 27 15:15:38.976131 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:38.976105    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cf85d6c8-kt6f4"
Jul 27 15:15:39.105568 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:39.105524    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-kt6f4"]
Jul 27 15:15:39.107958 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:15:39.107930    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11314a3b_d87e_4b19_86ed_034d1b9eb6be.slice/crio-be717f8c4dfc0275d414da49032a716ad028dd02886503f8965c38733bad0997 WatchSource:0}: Error finding container be717f8c4dfc0275d414da49032a716ad028dd02886503f8965c38733bad0997: Status 404 returned error can't find the container with id be717f8c4dfc0275d414da49032a716ad028dd02886503f8965c38733bad0997
Jul 27 15:15:39.732589 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:39.732545    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn" event={"ID":"3483f945-08fc-47e3-8c17-49c3be442ba4","Type":"ContainerStarted","Data":"4ff1265fbcf7362c1215bffdfe062f96a49b8e967921c395a9ac0f4c6fb982bc"}
Jul 27 15:15:39.733675 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:39.733653    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cf85d6c8-kt6f4" event={"ID":"11314a3b-d87e-4b19-86ed-034d1b9eb6be","Type":"ContainerStarted","Data":"be717f8c4dfc0275d414da49032a716ad028dd02886503f8965c38733bad0997"}
Jul 27 15:15:43.445802 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:43.445767    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:43.546733 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:43.546701    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:15:43.754607 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:43.754580    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn" event={"ID":"3483f945-08fc-47e3-8c17-49c3be442ba4","Type":"ContainerStarted","Data":"6f7f11ea31a8c5bb4f52bd1c818e9b4eb4fefcdf776842aa7cb0aebfee46fad5"}
Jul 27 15:15:43.754798 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:43.754740    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn"
Jul 27 15:15:43.756064 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:43.756040    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cf85d6c8-kt6f4" event={"ID":"11314a3b-d87e-4b19-86ed-034d1b9eb6be","Type":"ContainerStarted","Data":"698e843160f6e2ddb929cec764d89fe0fd6e7921761e8c074727ccddb277a034"}
Jul 27 15:15:43.756152 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:43.756098    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77cf85d6c8-kt6f4"
Jul 27 15:15:43.775153 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:43.775105    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn" podStartSLOduration=1.851584887 podStartE2EDuration="5.775092319s" totalImagesPullingTime="3.923507432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:15:43.773377656 +0000 UTC m=+615.202052840" watchObservedRunningTime="2026-07-27 15:15:43.775092319 +0000 UTC m=+615.203767528"
Jul 27 15:15:43.793470 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:43.793429    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77cf85d6c8-kt6f4" podStartSLOduration=2.0418159830000002 podStartE2EDuration="5.793415521s" totalImagesPullingTime="3.751599538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:15:43.792902395 +0000 UTC m=+615.221577578" watchObservedRunningTime="2026-07-27 15:15:43.793415521 +0000 UTC m=+615.222090704"
Jul 27 15:15:54.762334 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:54.762258    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn"
Jul 27 15:15:54.762334 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:54.762325    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77cf85d6c8-kt6f4"
Jul 27 15:15:54.828799 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:54.828768    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57b4cbfdf4-qcvqn"]
Jul 27 15:15:54.829032 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:54.829010    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn" podUID="3483f945-08fc-47e3-8c17-49c3be442ba4" containerName="manager" containerID="cri-o://6f7f11ea31a8c5bb4f52bd1c818e9b4eb4fefcdf776842aa7cb0aebfee46fad5" gracePeriod=30
Jul 27 15:15:55.079500 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.079473    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn"
Jul 27 15:15:55.146125 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.146093    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3483f945-08fc-47e3-8c17-49c3be442ba4-webhook-cert\") pod \"3483f945-08fc-47e3-8c17-49c3be442ba4\" (UID: \"3483f945-08fc-47e3-8c17-49c3be442ba4\") "
Jul 27 15:15:55.146299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.146184    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqz6s\" (UniqueName: \"kubernetes.io/projected/3483f945-08fc-47e3-8c17-49c3be442ba4-kube-api-access-sqz6s\") pod \"3483f945-08fc-47e3-8c17-49c3be442ba4\" (UID: \"3483f945-08fc-47e3-8c17-49c3be442ba4\") "
Jul 27 15:15:55.148337 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.148307    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3483f945-08fc-47e3-8c17-49c3be442ba4-kube-api-access-sqz6s" (OuterVolumeSpecName: "kube-api-access-sqz6s") pod "3483f945-08fc-47e3-8c17-49c3be442ba4" (UID: "3483f945-08fc-47e3-8c17-49c3be442ba4"). InnerVolumeSpecName "kube-api-access-sqz6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:15:55.148445 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.148311    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3483f945-08fc-47e3-8c17-49c3be442ba4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3483f945-08fc-47e3-8c17-49c3be442ba4" (UID: "3483f945-08fc-47e3-8c17-49c3be442ba4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:15:55.246937 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.246900    2548 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3483f945-08fc-47e3-8c17-49c3be442ba4-webhook-cert\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:15:55.246937 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.246928    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqz6s\" (UniqueName: \"kubernetes.io/projected/3483f945-08fc-47e3-8c17-49c3be442ba4-kube-api-access-sqz6s\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:15:55.808289 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.808255    2548 generic.go:356] "Generic (PLEG): container finished" podID="3483f945-08fc-47e3-8c17-49c3be442ba4" containerID="6f7f11ea31a8c5bb4f52bd1c818e9b4eb4fefcdf776842aa7cb0aebfee46fad5" exitCode=0
Jul 27 15:15:55.808699 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.808302    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn"
Jul 27 15:15:55.808699 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.808328    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn" event={"ID":"3483f945-08fc-47e3-8c17-49c3be442ba4","Type":"ContainerDied","Data":"6f7f11ea31a8c5bb4f52bd1c818e9b4eb4fefcdf776842aa7cb0aebfee46fad5"}
Jul 27 15:15:55.808699 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.808378    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b4cbfdf4-qcvqn" event={"ID":"3483f945-08fc-47e3-8c17-49c3be442ba4","Type":"ContainerDied","Data":"4ff1265fbcf7362c1215bffdfe062f96a49b8e967921c395a9ac0f4c6fb982bc"}
Jul 27 15:15:55.808699 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.808408    2548 scope.go:117] "RemoveContainer" containerID="6f7f11ea31a8c5bb4f52bd1c818e9b4eb4fefcdf776842aa7cb0aebfee46fad5"
Jul 27 15:15:55.817313 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.817297    2548 scope.go:117] "RemoveContainer" containerID="6f7f11ea31a8c5bb4f52bd1c818e9b4eb4fefcdf776842aa7cb0aebfee46fad5"
Jul 27 15:15:55.817578 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:15:55.817556    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f7f11ea31a8c5bb4f52bd1c818e9b4eb4fefcdf776842aa7cb0aebfee46fad5\": container with ID starting with 6f7f11ea31a8c5bb4f52bd1c818e9b4eb4fefcdf776842aa7cb0aebfee46fad5 not found: ID does not exist" containerID="6f7f11ea31a8c5bb4f52bd1c818e9b4eb4fefcdf776842aa7cb0aebfee46fad5"
Jul 27 15:15:55.817677 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.817585    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f7f11ea31a8c5bb4f52bd1c818e9b4eb4fefcdf776842aa7cb0aebfee46fad5"} err="failed to get container status \"6f7f11ea31a8c5bb4f52bd1c818e9b4eb4fefcdf776842aa7cb0aebfee46fad5\": rpc error: code = NotFound desc = could not find container \"6f7f11ea31a8c5bb4f52bd1c818e9b4eb4fefcdf776842aa7cb0aebfee46fad5\": container with ID starting with 6f7f11ea31a8c5bb4f52bd1c818e9b4eb4fefcdf776842aa7cb0aebfee46fad5 not found: ID does not exist"
Jul 27 15:15:55.830295 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.830271    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57b4cbfdf4-qcvqn"]
Jul 27 15:15:55.831711 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:55.831691    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57b4cbfdf4-qcvqn"]
Jul 27 15:15:57.139444 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:15:57.139410    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3483f945-08fc-47e3-8c17-49c3be442ba4" path="/var/lib/kubelet/pods/3483f945-08fc-47e3-8c17-49c3be442ba4/volumes"
Jul 27 15:16:14.695346 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:14.695316    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:16:14.792765 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:14.792740    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:16:15.366131 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.366097    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:16:15.499846 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.499818    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-574b54f485-9z998"]
Jul 27 15:16:15.500165 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.500154    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3483f945-08fc-47e3-8c17-49c3be442ba4" containerName="manager"
Jul 27 15:16:15.500206 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.500168    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="3483f945-08fc-47e3-8c17-49c3be442ba4" containerName="manager"
Jul 27 15:16:15.500240 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.500230    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="3483f945-08fc-47e3-8c17-49c3be442ba4" containerName="manager"
Jul 27 15:16:15.508240 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.508215    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998"
Jul 27 15:16:15.512244 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.512220    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 27 15:16:15.512390 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.512223    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-f826g\""
Jul 27 15:16:15.513920 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.513896    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-574b54f485-9z998"]
Jul 27 15:16:15.521343 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.521320    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ed8508b-568f-45f5-936c-eee83fe7734b-maas-api-tls\") pod \"maas-api-574b54f485-9z998\" (UID: \"2ed8508b-568f-45f5-936c-eee83fe7734b\") " pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998"
Jul 27 15:16:15.521423 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.521384    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ed8508b-568f-45f5-936c-eee83fe7734b-openshift-service-ca\") pod \"maas-api-574b54f485-9z998\" (UID: \"2ed8508b-568f-45f5-936c-eee83fe7734b\") " pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998"
Jul 27 15:16:15.521423 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.521412    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlgnh\" (UniqueName: \"kubernetes.io/projected/2ed8508b-568f-45f5-936c-eee83fe7734b-kube-api-access-hlgnh\") pod \"maas-api-574b54f485-9z998\" (UID: \"2ed8508b-568f-45f5-936c-eee83fe7734b\") " pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998"
Jul 27 15:16:15.622402 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.622326    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ed8508b-568f-45f5-936c-eee83fe7734b-maas-api-tls\") pod \"maas-api-574b54f485-9z998\" (UID: \"2ed8508b-568f-45f5-936c-eee83fe7734b\") " pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998"
Jul 27 15:16:15.622402 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.622371    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ed8508b-568f-45f5-936c-eee83fe7734b-openshift-service-ca\") pod \"maas-api-574b54f485-9z998\" (UID: \"2ed8508b-568f-45f5-936c-eee83fe7734b\") " pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998"
Jul 27 15:16:15.622402 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.622393    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlgnh\" (UniqueName: \"kubernetes.io/projected/2ed8508b-568f-45f5-936c-eee83fe7734b-kube-api-access-hlgnh\") pod \"maas-api-574b54f485-9z998\" (UID: \"2ed8508b-568f-45f5-936c-eee83fe7734b\") " pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998"
Jul 27 15:16:15.622743 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:16:15.622494    2548 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 27 15:16:15.622743 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:16:15.622596    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ed8508b-568f-45f5-936c-eee83fe7734b-maas-api-tls podName:2ed8508b-568f-45f5-936c-eee83fe7734b nodeName:}" failed. No retries permitted until 2026-07-27 15:16:16.122576626 +0000 UTC m=+647.551251788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2ed8508b-568f-45f5-936c-eee83fe7734b-maas-api-tls") pod "maas-api-574b54f485-9z998" (UID: "2ed8508b-568f-45f5-936c-eee83fe7734b") : secret "maas-api-serving-cert" not found
Jul 27 15:16:15.623066 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.623045    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ed8508b-568f-45f5-936c-eee83fe7734b-openshift-service-ca\") pod \"maas-api-574b54f485-9z998\" (UID: \"2ed8508b-568f-45f5-936c-eee83fe7734b\") " pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998"
Jul 27 15:16:15.631614 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.631583    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlgnh\" (UniqueName: \"kubernetes.io/projected/2ed8508b-568f-45f5-936c-eee83fe7734b-kube-api-access-hlgnh\") pod \"maas-api-574b54f485-9z998\" (UID: \"2ed8508b-568f-45f5-936c-eee83fe7734b\") " pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998"
Jul 27 15:16:15.746658 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.746630    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:16:15.854471 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.854441    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:16:15.950799 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:15.950711    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:16:16.023235 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.023200    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch"]
Jul 27 15:16:16.026903 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.026886    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch"
Jul 27 15:16:16.029911 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.029887    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-t2gzn\""
Jul 27 15:16:16.029911 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.029906    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 27 15:16:16.035338 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.035311    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch"]
Jul 27 15:16:16.128200 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.128167    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-hsfch\" (UID: \"4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch"
Jul 27 15:16:16.128367 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.128238    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ed8508b-568f-45f5-936c-eee83fe7734b-maas-api-tls\") pod \"maas-api-574b54f485-9z998\" (UID: \"2ed8508b-568f-45f5-936c-eee83fe7734b\") " pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998"
Jul 27 15:16:16.128367 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.128268    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd5nb\" (UniqueName: \"kubernetes.io/projected/4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8-kube-api-access-fd5nb\") pod \"payload-pre-processing-687dbb6cf6-hsfch\" (UID: \"4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch"
Jul 27 15:16:16.130839 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.130804    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ed8508b-568f-45f5-936c-eee83fe7734b-maas-api-tls\") pod \"maas-api-574b54f485-9z998\" (UID: \"2ed8508b-568f-45f5-936c-eee83fe7734b\") " pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998"
Jul 27 15:16:16.228992 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.228956    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd5nb\" (UniqueName: \"kubernetes.io/projected/4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8-kube-api-access-fd5nb\") pod \"payload-pre-processing-687dbb6cf6-hsfch\" (UID: \"4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch"
Jul 27 15:16:16.229156 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.229052    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-hsfch\" (UID: \"4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch"
Jul 27 15:16:16.229909 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.229882    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-hsfch\" (UID: \"4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch"
Jul 27 15:16:16.237382 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.237359    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd5nb\" (UniqueName: \"kubernetes.io/projected/4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8-kube-api-access-fd5nb\") pod \"payload-pre-processing-687dbb6cf6-hsfch\" (UID: \"4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch"
Jul 27 15:16:16.338571 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.338512    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch"
Jul 27 15:16:16.419693 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.419667    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998"
Jul 27 15:16:16.474689 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.474655    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch"]
Jul 27 15:16:16.479352 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:16:16.479284    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca7b7fd_d93a_4c9c_bef7_438f6b9249e8.slice/crio-568e9fc1b6c7c6713baf6232ca7c7f3ba2d34a7d009ffdf7cd73afb4b1a95362 WatchSource:0}: Error finding container 568e9fc1b6c7c6713baf6232ca7c7f3ba2d34a7d009ffdf7cd73afb4b1a95362: Status 404 returned error can't find the container with id 568e9fc1b6c7c6713baf6232ca7c7f3ba2d34a7d009ffdf7cd73afb4b1a95362
Jul 27 15:16:16.481675 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.481655    2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 15:16:16.571653 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.571630    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-574b54f485-9z998"]
Jul 27 15:16:16.573699 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:16:16.573671    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed8508b_568f_45f5_936c_eee83fe7734b.slice/crio-9d0927da2484d8685ffdbce53209e1813ae40db0eb86fb52d306207e99f62832 WatchSource:0}: Error finding container 9d0927da2484d8685ffdbce53209e1813ae40db0eb86fb52d306207e99f62832: Status 404 returned error can't find the container with id 9d0927da2484d8685ffdbce53209e1813ae40db0eb86fb52d306207e99f62832
Jul 27 15:16:16.907876 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.907792    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998" event={"ID":"2ed8508b-568f-45f5-936c-eee83fe7734b","Type":"ContainerStarted","Data":"9d0927da2484d8685ffdbce53209e1813ae40db0eb86fb52d306207e99f62832"}
Jul 27 15:16:16.908889 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:16.908862    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch" event={"ID":"4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8","Type":"ContainerStarted","Data":"568e9fc1b6c7c6713baf6232ca7c7f3ba2d34a7d009ffdf7cd73afb4b1a95362"}
Jul 27 15:16:19.925298 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:19.925259    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998" event={"ID":"2ed8508b-568f-45f5-936c-eee83fe7734b","Type":"ContainerStarted","Data":"e35cc614c5459c8af4512f65f6eb54798dbb4127bff2bb66cbcbc15a4cfbf8ea"}
Jul 27 15:16:19.925797 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:19.925328    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998"
Jul 27 15:16:19.926712 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:19.926693    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch" event={"ID":"4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8","Type":"ContainerStarted","Data":"7eb9e62005a1456b0c5288378e670e95d8c08e2b2f03796b60edd6fe23537458"}
Jul 27 15:16:19.926830 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:19.926818    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch"
Jul 27 15:16:19.943668 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:19.943623    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998" podStartSLOduration=2.540265432 podStartE2EDuration="4.943610549s" totalImagesPullingTime="2.403345117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:16:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:16:19.942041419 +0000 UTC m=+651.370716615" watchObservedRunningTime="2026-07-27 15:16:19.943610549 +0000 UTC m=+651.372285732"
Jul 27 15:16:19.959443 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:19.959403    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch" podStartSLOduration=1.456903705 podStartE2EDuration="3.959390845s" totalImagesPullingTime="2.50248714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:16:19.958026681 +0000 UTC m=+651.386701863" watchObservedRunningTime="2026-07-27 15:16:19.959390845 +0000 UTC m=+651.388066029"
Jul 27 15:16:25.936461 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:25.936432    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-574b54f485-9z998"
Jul 27 15:16:26.951425 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:26.951392    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:16:27.050339 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:27.050308    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:16:30.933649 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:30.933617    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-hsfch"
Jul 27 15:16:33.768162 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.768124    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"]
Jul 27 15:16:33.772777 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.772748    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.775895 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.775874    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 27 15:16:33.776716 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.776700    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-j4bbm\""
Jul 27 15:16:33.776817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.776792    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 27 15:16:33.776817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.776812    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 27 15:16:33.782738 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.782714    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"]
Jul 27 15:16:33.802559 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.802502    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:16:33.890597 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.890566    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.890773 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.890612    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.890773 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.890691    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.890773 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.890751    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.890904 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.890776    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4h7g\" (UniqueName: \"kubernetes.io/projected/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-kube-api-access-z4h7g\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.890904 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.890796    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.992248 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.992215    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4h7g\" (UniqueName: \"kubernetes.io/projected/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-kube-api-access-z4h7g\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.992248 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.992251    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.992476 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.992274    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.992476 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.992301    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.992476 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.992353    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.992476 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.992407    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.992773 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.992738    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.992901 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.992750    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.992901 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.992830    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.994799 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.994774    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:33.995082 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:33.995062    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:34.000513 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:34.000493    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4h7g\" (UniqueName: \"kubernetes.io/projected/04d3db51-1ae9-4e54-816c-c86f55b7d7d7-kube-api-access-z4h7g\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q\" (UID: \"04d3db51-1ae9-4e54-816c-c86f55b7d7d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:34.083917 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:34.083828    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:34.222726 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:34.222702    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"]
Jul 27 15:16:34.224806 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:16:34.224779    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d3db51_1ae9_4e54_816c_c86f55b7d7d7.slice/crio-43d782d727f1775cc1423341c8fe771234a42f22af9c302f4ebddee9e28b3f97 WatchSource:0}: Error finding container 43d782d727f1775cc1423341c8fe771234a42f22af9c302f4ebddee9e28b3f97: Status 404 returned error can't find the container with id 43d782d727f1775cc1423341c8fe771234a42f22af9c302f4ebddee9e28b3f97
Jul 27 15:16:34.995236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:34.995194    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q" event={"ID":"04d3db51-1ae9-4e54-816c-c86f55b7d7d7","Type":"ContainerStarted","Data":"43d782d727f1775cc1423341c8fe771234a42f22af9c302f4ebddee9e28b3f97"}
Jul 27 15:16:36.002848 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:36.002805    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q" event={"ID":"04d3db51-1ae9-4e54-816c-c86f55b7d7d7","Type":"ContainerStarted","Data":"1a6f014e2acb363b1664663b9deeb6ea1dd798b94c74333e0edbce24f5ce3333"}
Jul 27 15:16:36.021210 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:36.021159    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q" podStartSLOduration=1.777181935 podStartE2EDuration="3.021142572s" totalImagesPullingTime="1.243960637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:16:36.020223319 +0000 UTC m=+667.448898508" watchObservedRunningTime="2026-07-27 15:16:36.021142572 +0000 UTC m=+667.449817754"
Jul 27 15:16:43.696898 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:43.696849    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:16:44.084525 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:44.084487    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:44.084753 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:44.084589    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:44.097087 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:44.097055    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:45.060333 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:45.060306    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q"
Jul 27 15:16:53.673857 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.673825    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"]
Jul 27 15:16:53.677432 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.677416    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.680569 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.680519    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 27 15:16:53.692219 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.692199    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"]
Jul 27 15:16:53.700706 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.700683    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:16:53.873203 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.873163    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lczrf\" (UniqueName: \"kubernetes.io/projected/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-kube-api-access-lczrf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.873373 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.873215    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.873373 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.873243    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.873480 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.873364    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.873480 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.873419    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.873625 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.873521    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.974965 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.974872    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.974965 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.974926    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.975193 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.974974    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lczrf\" (UniqueName: \"kubernetes.io/projected/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-kube-api-access-lczrf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.975193 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.975009    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.975193 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.975033    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.975193 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.975103    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.975434 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.975367    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.975434 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.975412    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.975550 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.975454    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.977305 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.977281    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.977559 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.977522    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.983802 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.983778    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lczrf\" (UniqueName: \"kubernetes.io/projected/bc6468e1-d6e0-4fdd-bb81-1b88d5216910-kube-api-access-lczrf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7444p\" (UID: \"bc6468e1-d6e0-4fdd-bb81-1b88d5216910\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:53.987680 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:53.987655    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:16:54.129303 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:54.129279    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"]
Jul 27 15:16:54.131396 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:16:54.131360    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc6468e1_d6e0_4fdd_bb81_1b88d5216910.slice/crio-cec041a0fe5522643090d4c10f0dc96bcc483471f1eccd85da5f4f91bcabe00d WatchSource:0}: Error finding container cec041a0fe5522643090d4c10f0dc96bcc483471f1eccd85da5f4f91bcabe00d: Status 404 returned error can't find the container with id cec041a0fe5522643090d4c10f0dc96bcc483471f1eccd85da5f4f91bcabe00d
Jul 27 15:16:55.096597 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:55.096559    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p" event={"ID":"bc6468e1-d6e0-4fdd-bb81-1b88d5216910","Type":"ContainerStarted","Data":"2ee863ad33187c0a73bc31f0a2c60e090c76a512120b8361b3a3a31d7e6e16ab"}
Jul 27 15:16:55.097093 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:55.096622    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p" event={"ID":"bc6468e1-d6e0-4fdd-bb81-1b88d5216910","Type":"ContainerStarted","Data":"cec041a0fe5522643090d4c10f0dc96bcc483471f1eccd85da5f4f91bcabe00d"}
Jul 27 15:16:55.115330 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:16:55.115280    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p" podStartSLOduration=1.8523322009999998 podStartE2EDuration="2.115263664s" totalImagesPullingTime="262.931463ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:16:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:16:55.113848285 +0000 UTC m=+686.542523470" watchObservedRunningTime="2026-07-27 15:16:55.115263664 +0000 UTC m=+686.543938869"
Jul 27 15:17:01.881054 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:01.881019    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:17:03.988652 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:03.988612    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:17:03.989046 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:03.988683    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:17:04.001203 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:04.001181    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:17:04.154553 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:04.154505    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7444p"
Jul 27 15:17:16.082618 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.082575    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"]
Jul 27 15:17:16.088449 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.088428    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.091293 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.091278    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 27 15:17:16.097410 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.097386    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"]
Jul 27 15:17:16.158549 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.158492    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:17:16.180291 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.180261    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2c26f9f-92dd-447c-a388-8078c36b9b17-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.180291 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.180296    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2c26f9f-92dd-447c-a388-8078c36b9b17-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.180490 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.180317    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2c26f9f-92dd-447c-a388-8078c36b9b17-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.180490 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.180409    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2c26f9f-92dd-447c-a388-8078c36b9b17-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.180490 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.180461    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52tc2\" (UniqueName: \"kubernetes.io/projected/c2c26f9f-92dd-447c-a388-8078c36b9b17-kube-api-access-52tc2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.180620 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.180506    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2c26f9f-92dd-447c-a388-8078c36b9b17-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.281551 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.281506    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2c26f9f-92dd-447c-a388-8078c36b9b17-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.281739 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.281587    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2c26f9f-92dd-447c-a388-8078c36b9b17-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.281739 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.281609    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2c26f9f-92dd-447c-a388-8078c36b9b17-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.281739 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.281628    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2c26f9f-92dd-447c-a388-8078c36b9b17-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.281739 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.281681    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2c26f9f-92dd-447c-a388-8078c36b9b17-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.281739 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.281726    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52tc2\" (UniqueName: \"kubernetes.io/projected/c2c26f9f-92dd-447c-a388-8078c36b9b17-kube-api-access-52tc2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.282019 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.281972    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2c26f9f-92dd-447c-a388-8078c36b9b17-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.282091 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.282074    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2c26f9f-92dd-447c-a388-8078c36b9b17-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.282161 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.282145    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2c26f9f-92dd-447c-a388-8078c36b9b17-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.284133 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.284106    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2c26f9f-92dd-447c-a388-8078c36b9b17-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.284389 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.284370    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2c26f9f-92dd-447c-a388-8078c36b9b17-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.289976 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.289954    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52tc2\" (UniqueName: \"kubernetes.io/projected/c2c26f9f-92dd-447c-a388-8078c36b9b17-kube-api-access-52tc2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cd8tx\" (UID: \"c2c26f9f-92dd-447c-a388-8078c36b9b17\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.399635 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.399521    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:16.533391 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:16.533368    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"]
Jul 27 15:17:16.536508 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:17:16.536470    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2c26f9f_92dd_447c_a388_8078c36b9b17.slice/crio-0a4b3da1a9477f18edd945655e2b634a7cd7dc08b1067228c324db7352535fec WatchSource:0}: Error finding container 0a4b3da1a9477f18edd945655e2b634a7cd7dc08b1067228c324db7352535fec: Status 404 returned error can't find the container with id 0a4b3da1a9477f18edd945655e2b634a7cd7dc08b1067228c324db7352535fec
Jul 27 15:17:17.202066 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:17.201973    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx" event={"ID":"c2c26f9f-92dd-447c-a388-8078c36b9b17","Type":"ContainerStarted","Data":"292c9cfa50144651e32fe7acf8b6946b3c2c2a1e96ec7963f027df46bc945dcc"}
Jul 27 15:17:17.202066 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:17.202035    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx" event={"ID":"c2c26f9f-92dd-447c-a388-8078c36b9b17","Type":"ContainerStarted","Data":"0a4b3da1a9477f18edd945655e2b634a7cd7dc08b1067228c324db7352535fec"}
Jul 27 15:17:17.222545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:17.222484    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx" podStartSLOduration=1.006308757 podStartE2EDuration="1.222469767s" totalImagesPullingTime="216.16101ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:17:17.221572701 +0000 UTC m=+708.650247881" watchObservedRunningTime="2026-07-27 15:17:17.222469767 +0000 UTC m=+708.651144998"
Jul 27 15:17:18.406384 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:18.406348    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:17:26.400391 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:26.400349    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:26.400952 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:26.400427    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:26.412705 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:26.412682    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:27.260111 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:27.260082    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cd8tx"
Jul 27 15:17:33.377333 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:33.377291    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:17:34.176983 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:34.176946    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:17:51.103196 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:17:51.103163    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:18:48.009549 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:18:48.009502    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:18:49.157588 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:18:49.157506    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:18:49.254687 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:18:49.254645    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:19:36.930817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:36.930783    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-kt6f4"]
Jul 27 15:19:36.931273 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:36.931054    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-77cf85d6c8-kt6f4" podUID="11314a3b-d87e-4b19-86ed-034d1b9eb6be" containerName="manager" containerID="cri-o://698e843160f6e2ddb929cec764d89fe0fd6e7921761e8c074727ccddb277a034" gracePeriod=30
Jul 27 15:19:37.180792 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.180766    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cf85d6c8-kt6f4"
Jul 27 15:19:37.254547 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.254502    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjhk2\" (UniqueName: \"kubernetes.io/projected/11314a3b-d87e-4b19-86ed-034d1b9eb6be-kube-api-access-zjhk2\") pod \"11314a3b-d87e-4b19-86ed-034d1b9eb6be\" (UID: \"11314a3b-d87e-4b19-86ed-034d1b9eb6be\") "
Jul 27 15:19:37.254729 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.254599    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11314a3b-d87e-4b19-86ed-034d1b9eb6be-webhook-cert\") pod \"11314a3b-d87e-4b19-86ed-034d1b9eb6be\" (UID: \"11314a3b-d87e-4b19-86ed-034d1b9eb6be\") "
Jul 27 15:19:37.256694 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.256665    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11314a3b-d87e-4b19-86ed-034d1b9eb6be-kube-api-access-zjhk2" (OuterVolumeSpecName: "kube-api-access-zjhk2") pod "11314a3b-d87e-4b19-86ed-034d1b9eb6be" (UID: "11314a3b-d87e-4b19-86ed-034d1b9eb6be"). InnerVolumeSpecName "kube-api-access-zjhk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:19:37.256838 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.256815    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11314a3b-d87e-4b19-86ed-034d1b9eb6be-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "11314a3b-d87e-4b19-86ed-034d1b9eb6be" (UID: "11314a3b-d87e-4b19-86ed-034d1b9eb6be"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:19:37.355951 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.355918    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjhk2\" (UniqueName: \"kubernetes.io/projected/11314a3b-d87e-4b19-86ed-034d1b9eb6be-kube-api-access-zjhk2\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:19:37.355951 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.355946    2548 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11314a3b-d87e-4b19-86ed-034d1b9eb6be-webhook-cert\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:19:37.871505 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.871472    2548 generic.go:356] "Generic (PLEG): container finished" podID="11314a3b-d87e-4b19-86ed-034d1b9eb6be" containerID="698e843160f6e2ddb929cec764d89fe0fd6e7921761e8c074727ccddb277a034" exitCode=0
Jul 27 15:19:37.871733 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.871522    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cf85d6c8-kt6f4"
Jul 27 15:19:37.871733 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.871561    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cf85d6c8-kt6f4" event={"ID":"11314a3b-d87e-4b19-86ed-034d1b9eb6be","Type":"ContainerDied","Data":"698e843160f6e2ddb929cec764d89fe0fd6e7921761e8c074727ccddb277a034"}
Jul 27 15:19:37.871733 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.871617    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cf85d6c8-kt6f4" event={"ID":"11314a3b-d87e-4b19-86ed-034d1b9eb6be","Type":"ContainerDied","Data":"be717f8c4dfc0275d414da49032a716ad028dd02886503f8965c38733bad0997"}
Jul 27 15:19:37.871733 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.871638    2548 scope.go:117] "RemoveContainer" containerID="698e843160f6e2ddb929cec764d89fe0fd6e7921761e8c074727ccddb277a034"
Jul 27 15:19:37.881203 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.881185    2548 scope.go:117] "RemoveContainer" containerID="698e843160f6e2ddb929cec764d89fe0fd6e7921761e8c074727ccddb277a034"
Jul 27 15:19:37.881481 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:19:37.881462    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"698e843160f6e2ddb929cec764d89fe0fd6e7921761e8c074727ccddb277a034\": container with ID starting with 698e843160f6e2ddb929cec764d89fe0fd6e7921761e8c074727ccddb277a034 not found: ID does not exist" containerID="698e843160f6e2ddb929cec764d89fe0fd6e7921761e8c074727ccddb277a034"
Jul 27 15:19:37.881555 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.881494    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"698e843160f6e2ddb929cec764d89fe0fd6e7921761e8c074727ccddb277a034"} err="failed to get container status \"698e843160f6e2ddb929cec764d89fe0fd6e7921761e8c074727ccddb277a034\": rpc error: code = NotFound desc = could not find container \"698e843160f6e2ddb929cec764d89fe0fd6e7921761e8c074727ccddb277a034\": container with ID starting with 698e843160f6e2ddb929cec764d89fe0fd6e7921761e8c074727ccddb277a034 not found: ID does not exist"
Jul 27 15:19:37.894456 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.894426    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-kt6f4"]
Jul 27 15:19:37.896188 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:37.896169    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-kt6f4"]
Jul 27 15:19:39.138774 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:19:39.138744    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11314a3b-d87e-4b19-86ed-034d1b9eb6be" path="/var/lib/kubelet/pods/11314a3b-d87e-4b19-86ed-034d1b9eb6be/volumes"
Jul 27 15:20:19.701120 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:19.701037    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:20:38.727432 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.727397    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-4l5rl"]
Jul 27 15:20:38.729687 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.727812    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11314a3b-d87e-4b19-86ed-034d1b9eb6be" containerName="manager"
Jul 27 15:20:38.729687 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.727825    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="11314a3b-d87e-4b19-86ed-034d1b9eb6be" containerName="manager"
Jul 27 15:20:38.729687 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.727923    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="11314a3b-d87e-4b19-86ed-034d1b9eb6be" containerName="manager"
Jul 27 15:20:38.730767 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.730751    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cf85d6c8-4l5rl"
Jul 27 15:20:38.733795 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.733773    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 15:20:38.734957 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.734932    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tbchg\""
Jul 27 15:20:38.735068 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.735037    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 27 15:20:38.739799 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.739772    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-4l5rl"]
Jul 27 15:20:38.822263 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.822223    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsqhp\" (UniqueName: \"kubernetes.io/projected/779577ec-8e41-4761-9ad3-fa91eaee9610-kube-api-access-dsqhp\") pod \"maas-controller-77cf85d6c8-4l5rl\" (UID: \"779577ec-8e41-4761-9ad3-fa91eaee9610\") " pod="opendatahub/maas-controller-77cf85d6c8-4l5rl"
Jul 27 15:20:38.822454 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.822304    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/779577ec-8e41-4761-9ad3-fa91eaee9610-webhook-cert\") pod \"maas-controller-77cf85d6c8-4l5rl\" (UID: \"779577ec-8e41-4761-9ad3-fa91eaee9610\") " pod="opendatahub/maas-controller-77cf85d6c8-4l5rl"
Jul 27 15:20:38.923310 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.923260    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/779577ec-8e41-4761-9ad3-fa91eaee9610-webhook-cert\") pod \"maas-controller-77cf85d6c8-4l5rl\" (UID: \"779577ec-8e41-4761-9ad3-fa91eaee9610\") " pod="opendatahub/maas-controller-77cf85d6c8-4l5rl"
Jul 27 15:20:38.923515 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.923373    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsqhp\" (UniqueName: \"kubernetes.io/projected/779577ec-8e41-4761-9ad3-fa91eaee9610-kube-api-access-dsqhp\") pod \"maas-controller-77cf85d6c8-4l5rl\" (UID: \"779577ec-8e41-4761-9ad3-fa91eaee9610\") " pod="opendatahub/maas-controller-77cf85d6c8-4l5rl"
Jul 27 15:20:38.926096 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.926072    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/779577ec-8e41-4761-9ad3-fa91eaee9610-webhook-cert\") pod \"maas-controller-77cf85d6c8-4l5rl\" (UID: \"779577ec-8e41-4761-9ad3-fa91eaee9610\") " pod="opendatahub/maas-controller-77cf85d6c8-4l5rl"
Jul 27 15:20:38.932140 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:38.932101    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsqhp\" (UniqueName: \"kubernetes.io/projected/779577ec-8e41-4761-9ad3-fa91eaee9610-kube-api-access-dsqhp\") pod \"maas-controller-77cf85d6c8-4l5rl\" (UID: \"779577ec-8e41-4761-9ad3-fa91eaee9610\") " pod="opendatahub/maas-controller-77cf85d6c8-4l5rl"
Jul 27 15:20:39.042733 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:39.042632    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cf85d6c8-4l5rl"
Jul 27 15:20:39.183429 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:39.183305    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-4l5rl"]
Jul 27 15:20:39.186134 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:20:39.186106    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod779577ec_8e41_4761_9ad3_fa91eaee9610.slice/crio-7080fc9eb96943ef7fc968d5c6011c2319e8823ddff2fea57762a663518cd038 WatchSource:0}: Error finding container 7080fc9eb96943ef7fc968d5c6011c2319e8823ddff2fea57762a663518cd038: Status 404 returned error can't find the container with id 7080fc9eb96943ef7fc968d5c6011c2319e8823ddff2fea57762a663518cd038
Jul 27 15:20:40.166109 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:40.166063    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cf85d6c8-4l5rl" event={"ID":"779577ec-8e41-4761-9ad3-fa91eaee9610","Type":"ContainerStarted","Data":"055d803da97f1d3ff1af673cbc385828a44bd92490dc22fe350610a090b8e4d5"}
Jul 27 15:20:40.166520 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:40.166166    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cf85d6c8-4l5rl" event={"ID":"779577ec-8e41-4761-9ad3-fa91eaee9610","Type":"ContainerStarted","Data":"7080fc9eb96943ef7fc968d5c6011c2319e8823ddff2fea57762a663518cd038"}
Jul 27 15:20:40.166520 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:40.166195    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77cf85d6c8-4l5rl"
Jul 27 15:20:40.183962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:40.183903    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77cf85d6c8-4l5rl" podStartSLOduration=1.3650995369999999 podStartE2EDuration="2.183884467s" totalImagesPullingTime="818.78493ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:20:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:20:40.182699619 +0000 UTC m=+911.611374802" watchObservedRunningTime="2026-07-27 15:20:40.183884467 +0000 UTC m=+911.612559650"
Jul 27 15:20:45.508363 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:45.508290    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:20:45.562504 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:45.562471    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:20:51.173841 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:51.173815    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77cf85d6c8-4l5rl"
Jul 27 15:20:55.405004 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:55.404966    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:20:55.733547 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:55.733499    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:20:55.820110 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:55.820071    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-4l5rl"]
Jul 27 15:20:55.820341 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:55.820313    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-77cf85d6c8-4l5rl" podUID="779577ec-8e41-4761-9ad3-fa91eaee9610" containerName="manager" containerID="cri-o://055d803da97f1d3ff1af673cbc385828a44bd92490dc22fe350610a090b8e4d5" gracePeriod=30
Jul 27 15:20:56.070043 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.070017    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cf85d6c8-4l5rl"
Jul 27 15:20:56.181426 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.181393    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsqhp\" (UniqueName: \"kubernetes.io/projected/779577ec-8e41-4761-9ad3-fa91eaee9610-kube-api-access-dsqhp\") pod \"779577ec-8e41-4761-9ad3-fa91eaee9610\" (UID: \"779577ec-8e41-4761-9ad3-fa91eaee9610\") "
Jul 27 15:20:56.181616 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.181525    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/779577ec-8e41-4761-9ad3-fa91eaee9610-webhook-cert\") pod \"779577ec-8e41-4761-9ad3-fa91eaee9610\" (UID: \"779577ec-8e41-4761-9ad3-fa91eaee9610\") "
Jul 27 15:20:56.183754 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.183721    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/779577ec-8e41-4761-9ad3-fa91eaee9610-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "779577ec-8e41-4761-9ad3-fa91eaee9610" (UID: "779577ec-8e41-4761-9ad3-fa91eaee9610"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:20:56.183850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.183751    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/779577ec-8e41-4761-9ad3-fa91eaee9610-kube-api-access-dsqhp" (OuterVolumeSpecName: "kube-api-access-dsqhp") pod "779577ec-8e41-4761-9ad3-fa91eaee9610" (UID: "779577ec-8e41-4761-9ad3-fa91eaee9610"). InnerVolumeSpecName "kube-api-access-dsqhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:20:56.240497 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.240397    2548 generic.go:356] "Generic (PLEG): container finished" podID="779577ec-8e41-4761-9ad3-fa91eaee9610" containerID="055d803da97f1d3ff1af673cbc385828a44bd92490dc22fe350610a090b8e4d5" exitCode=0
Jul 27 15:20:56.240497 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.240447    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cf85d6c8-4l5rl"
Jul 27 15:20:56.240766 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.240446    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cf85d6c8-4l5rl" event={"ID":"779577ec-8e41-4761-9ad3-fa91eaee9610","Type":"ContainerDied","Data":"055d803da97f1d3ff1af673cbc385828a44bd92490dc22fe350610a090b8e4d5"}
Jul 27 15:20:56.240766 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.240618    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cf85d6c8-4l5rl" event={"ID":"779577ec-8e41-4761-9ad3-fa91eaee9610","Type":"ContainerDied","Data":"7080fc9eb96943ef7fc968d5c6011c2319e8823ddff2fea57762a663518cd038"}
Jul 27 15:20:56.240766 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.240654    2548 scope.go:117] "RemoveContainer" containerID="055d803da97f1d3ff1af673cbc385828a44bd92490dc22fe350610a090b8e4d5"
Jul 27 15:20:56.250228 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.250212    2548 scope.go:117] "RemoveContainer" containerID="055d803da97f1d3ff1af673cbc385828a44bd92490dc22fe350610a090b8e4d5"
Jul 27 15:20:56.250466 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:20:56.250447    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"055d803da97f1d3ff1af673cbc385828a44bd92490dc22fe350610a090b8e4d5\": container with ID starting with 055d803da97f1d3ff1af673cbc385828a44bd92490dc22fe350610a090b8e4d5 not found: ID does not exist" containerID="055d803da97f1d3ff1af673cbc385828a44bd92490dc22fe350610a090b8e4d5"
Jul 27 15:20:56.250510 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.250475    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"055d803da97f1d3ff1af673cbc385828a44bd92490dc22fe350610a090b8e4d5"} err="failed to get container status \"055d803da97f1d3ff1af673cbc385828a44bd92490dc22fe350610a090b8e4d5\": rpc error: code = NotFound desc = could not find container \"055d803da97f1d3ff1af673cbc385828a44bd92490dc22fe350610a090b8e4d5\": container with ID starting with 055d803da97f1d3ff1af673cbc385828a44bd92490dc22fe350610a090b8e4d5 not found: ID does not exist"
Jul 27 15:20:56.264019 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.263994    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-4l5rl"]
Jul 27 15:20:56.267627 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.267601    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-4l5rl"]
Jul 27 15:20:56.283059 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.283037    2548 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/779577ec-8e41-4761-9ad3-fa91eaee9610-webhook-cert\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:20:56.283059 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.283057    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsqhp\" (UniqueName: \"kubernetes.io/projected/779577ec-8e41-4761-9ad3-fa91eaee9610-kube-api-access-dsqhp\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:20:56.949261 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:56.949225    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:20:57.062650 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.062619    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-bm466"]
Jul 27 15:20:57.063036 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.063022    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="779577ec-8e41-4761-9ad3-fa91eaee9610" containerName="manager"
Jul 27 15:20:57.063087 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.063038    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="779577ec-8e41-4761-9ad3-fa91eaee9610" containerName="manager"
Jul 27 15:20:57.063156 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.063147    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="779577ec-8e41-4761-9ad3-fa91eaee9610" containerName="manager"
Jul 27 15:20:57.067495 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.067478    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cf85d6c8-bm466"
Jul 27 15:20:57.072274 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.071740    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 27 15:20:57.072274 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.071799    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 15:20:57.072274 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.071915    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tbchg\""
Jul 27 15:20:57.080403 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.076632    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-bm466"]
Jul 27 15:20:57.138598 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.138555    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="779577ec-8e41-4761-9ad3-fa91eaee9610" path="/var/lib/kubelet/pods/779577ec-8e41-4761-9ad3-fa91eaee9610/volumes"
Jul 27 15:20:57.192241 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.192205    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/403e2717-a834-46a4-b36a-29e220418ad9-webhook-cert\") pod \"maas-controller-77cf85d6c8-bm466\" (UID: \"403e2717-a834-46a4-b36a-29e220418ad9\") " pod="opendatahub/maas-controller-77cf85d6c8-bm466"
Jul 27 15:20:57.192241 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.192249    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h2ng\" (UniqueName: \"kubernetes.io/projected/403e2717-a834-46a4-b36a-29e220418ad9-kube-api-access-4h2ng\") pod \"maas-controller-77cf85d6c8-bm466\" (UID: \"403e2717-a834-46a4-b36a-29e220418ad9\") " pod="opendatahub/maas-controller-77cf85d6c8-bm466"
Jul 27 15:20:57.293893 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.293856    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/403e2717-a834-46a4-b36a-29e220418ad9-webhook-cert\") pod \"maas-controller-77cf85d6c8-bm466\" (UID: \"403e2717-a834-46a4-b36a-29e220418ad9\") " pod="opendatahub/maas-controller-77cf85d6c8-bm466"
Jul 27 15:20:57.293893 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.293895    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h2ng\" (UniqueName: \"kubernetes.io/projected/403e2717-a834-46a4-b36a-29e220418ad9-kube-api-access-4h2ng\") pod \"maas-controller-77cf85d6c8-bm466\" (UID: \"403e2717-a834-46a4-b36a-29e220418ad9\") " pod="opendatahub/maas-controller-77cf85d6c8-bm466"
Jul 27 15:20:57.296658 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.296624    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/403e2717-a834-46a4-b36a-29e220418ad9-webhook-cert\") pod \"maas-controller-77cf85d6c8-bm466\" (UID: \"403e2717-a834-46a4-b36a-29e220418ad9\") " pod="opendatahub/maas-controller-77cf85d6c8-bm466"
Jul 27 15:20:57.302919 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.302897    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h2ng\" (UniqueName: \"kubernetes.io/projected/403e2717-a834-46a4-b36a-29e220418ad9-kube-api-access-4h2ng\") pod \"maas-controller-77cf85d6c8-bm466\" (UID: \"403e2717-a834-46a4-b36a-29e220418ad9\") " pod="opendatahub/maas-controller-77cf85d6c8-bm466"
Jul 27 15:20:57.381252 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.381219    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cf85d6c8-bm466"
Jul 27 15:20:57.518988 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:57.518854    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77cf85d6c8-bm466"]
Jul 27 15:20:57.521674 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:20:57.521641    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod403e2717_a834_46a4_b36a_29e220418ad9.slice/crio-d165ecf08c9977ef2178871445d7b050cc02b07048a33e95d53b16e606f3c007 WatchSource:0}: Error finding container d165ecf08c9977ef2178871445d7b050cc02b07048a33e95d53b16e606f3c007: Status 404 returned error can't find the container with id d165ecf08c9977ef2178871445d7b050cc02b07048a33e95d53b16e606f3c007
Jul 27 15:20:58.255996 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:58.255959    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cf85d6c8-bm466" event={"ID":"403e2717-a834-46a4-b36a-29e220418ad9","Type":"ContainerStarted","Data":"34e9be41b79adc806d515b458992f4840e14aca9d0cc862e58b3225109de6ff7"}
Jul 27 15:20:58.256419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:58.256009    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cf85d6c8-bm466" event={"ID":"403e2717-a834-46a4-b36a-29e220418ad9","Type":"ContainerStarted","Data":"d165ecf08c9977ef2178871445d7b050cc02b07048a33e95d53b16e606f3c007"}
Jul 27 15:20:58.256419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:58.256086    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77cf85d6c8-bm466"
Jul 27 15:20:58.274722 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:20:58.274680    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77cf85d6c8-bm466" podStartSLOduration=0.934618161 podStartE2EDuration="1.274660932s" totalImagesPullingTime="340.042771ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:20:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:20:58.274343733 +0000 UTC m=+929.703018929" watchObservedRunningTime="2026-07-27 15:20:58.274660932 +0000 UTC m=+929.703336116"
Jul 27 15:21:09.263902 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:21:09.263869    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77cf85d6c8-bm466"
Jul 27 15:21:32.612664 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:21:32.612615    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:21:32.697523 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:21:32.697483    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:21:32.898756 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:21:32.898653    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:21:38.814854 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:21:38.814818    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:21:38.919830 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:21:38.919791    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:21:39.017753 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:21:39.017714    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:21:40.208247 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:21:40.208214    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:21:45.087053 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:21:45.087007    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:21:45.186578 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:21:45.186504    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:22:55.787968 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:22:55.787929    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:23:21.486212 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:23:21.486130    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:24:33.492417 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:24:33.492373    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:25:42.524802 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:42.524768    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"]
Jul 27 15:25:42.527403 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:42.525039    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm" podUID="b70a3472-e01f-4166-88dc-d38fb2f4ff0b" containerName="manager" containerID="cri-o://b899851c0301fabec51585ad0100b2a435ae5ed24fe116b010bd5af714a421e8" gracePeriod=10
Jul 27 15:25:45.636745 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:45.636709    2548 generic.go:356] "Generic (PLEG): container finished" podID="b70a3472-e01f-4166-88dc-d38fb2f4ff0b" containerID="b899851c0301fabec51585ad0100b2a435ae5ed24fe116b010bd5af714a421e8" exitCode=0
Jul 27 15:25:45.637230 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:45.636778    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm" event={"ID":"b70a3472-e01f-4166-88dc-d38fb2f4ff0b","Type":"ContainerDied","Data":"b899851c0301fabec51585ad0100b2a435ae5ed24fe116b010bd5af714a421e8"}
Jul 27 15:25:45.676552 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:45.676518    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"
Jul 27 15:25:45.777866 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:45.777830    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk95k\" (UniqueName: \"kubernetes.io/projected/b70a3472-e01f-4166-88dc-d38fb2f4ff0b-kube-api-access-hk95k\") pod \"b70a3472-e01f-4166-88dc-d38fb2f4ff0b\" (UID: \"b70a3472-e01f-4166-88dc-d38fb2f4ff0b\") "
Jul 27 15:25:45.778019 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:45.777898    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b70a3472-e01f-4166-88dc-d38fb2f4ff0b-extensions-socket-volume\") pod \"b70a3472-e01f-4166-88dc-d38fb2f4ff0b\" (UID: \"b70a3472-e01f-4166-88dc-d38fb2f4ff0b\") "
Jul 27 15:25:45.778221 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:45.778199    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70a3472-e01f-4166-88dc-d38fb2f4ff0b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b70a3472-e01f-4166-88dc-d38fb2f4ff0b" (UID: "b70a3472-e01f-4166-88dc-d38fb2f4ff0b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 15:25:45.779971 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:45.779949    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b70a3472-e01f-4166-88dc-d38fb2f4ff0b-kube-api-access-hk95k" (OuterVolumeSpecName: "kube-api-access-hk95k") pod "b70a3472-e01f-4166-88dc-d38fb2f4ff0b" (UID: "b70a3472-e01f-4166-88dc-d38fb2f4ff0b"). InnerVolumeSpecName "kube-api-access-hk95k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:25:45.878825 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:45.878757    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk95k\" (UniqueName: \"kubernetes.io/projected/b70a3472-e01f-4166-88dc-d38fb2f4ff0b-kube-api-access-hk95k\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:25:45.878825 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:45.878781    2548 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b70a3472-e01f-4166-88dc-d38fb2f4ff0b-extensions-socket-volume\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:25:46.642859 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:46.642831    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"
Jul 27 15:25:46.643292 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:46.642826    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm" event={"ID":"b70a3472-e01f-4166-88dc-d38fb2f4ff0b","Type":"ContainerDied","Data":"520d6a5b052a0455d7bd4e18025a4c5f83640dbd1f0a4bbf182e77abf3fbf63d"}
Jul 27 15:25:46.643292 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:46.642965    2548 scope.go:117] "RemoveContainer" containerID="b899851c0301fabec51585ad0100b2a435ae5ed24fe116b010bd5af714a421e8"
Jul 27 15:25:46.666610 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:46.666586    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"]
Jul 27 15:25:46.671419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:46.671371    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q6mbm"]
Jul 27 15:25:47.138835 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:25:47.138803    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b70a3472-e01f-4166-88dc-d38fb2f4ff0b" path="/var/lib/kubelet/pods/b70a3472-e01f-4166-88dc-d38fb2f4ff0b/volumes"
Jul 27 15:26:43.688783 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:43.688748    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg"]
Jul 27 15:26:43.689239 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:43.689119    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b70a3472-e01f-4166-88dc-d38fb2f4ff0b" containerName="manager"
Jul 27 15:26:43.689239 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:43.689130    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="b70a3472-e01f-4166-88dc-d38fb2f4ff0b" containerName="manager"
Jul 27 15:26:43.689239 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:43.689208    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="b70a3472-e01f-4166-88dc-d38fb2f4ff0b" containerName="manager"
Jul 27 15:26:43.693586 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:43.693565    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg"
Jul 27 15:26:43.696453 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:43.696434    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rqmqs\""
Jul 27 15:26:43.708222 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:43.708190    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg"]
Jul 27 15:26:43.777135 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:43.777103    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0eec5852-d33e-40f7-8d1f-43e7d7eead44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jqpfg\" (UID: \"0eec5852-d33e-40f7-8d1f-43e7d7eead44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg"
Jul 27 15:26:43.777290 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:43.777160    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxpqq\" (UniqueName: \"kubernetes.io/projected/0eec5852-d33e-40f7-8d1f-43e7d7eead44-kube-api-access-fxpqq\") pod \"kuadrant-operator-controller-manager-86878c769f-jqpfg\" (UID: \"0eec5852-d33e-40f7-8d1f-43e7d7eead44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg"
Jul 27 15:26:43.877799 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:43.877769    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxpqq\" (UniqueName: \"kubernetes.io/projected/0eec5852-d33e-40f7-8d1f-43e7d7eead44-kube-api-access-fxpqq\") pod \"kuadrant-operator-controller-manager-86878c769f-jqpfg\" (UID: \"0eec5852-d33e-40f7-8d1f-43e7d7eead44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg"
Jul 27 15:26:43.877989 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:43.877841    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0eec5852-d33e-40f7-8d1f-43e7d7eead44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jqpfg\" (UID: \"0eec5852-d33e-40f7-8d1f-43e7d7eead44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg"
Jul 27 15:26:43.878151 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:43.878132    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0eec5852-d33e-40f7-8d1f-43e7d7eead44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jqpfg\" (UID: \"0eec5852-d33e-40f7-8d1f-43e7d7eead44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg"
Jul 27 15:26:43.886799 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:43.886777    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxpqq\" (UniqueName: \"kubernetes.io/projected/0eec5852-d33e-40f7-8d1f-43e7d7eead44-kube-api-access-fxpqq\") pod \"kuadrant-operator-controller-manager-86878c769f-jqpfg\" (UID: \"0eec5852-d33e-40f7-8d1f-43e7d7eead44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg"
Jul 27 15:26:44.003663 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:44.003632    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg"
Jul 27 15:26:44.165860 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:44.165777    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg"]
Jul 27 15:26:44.168307 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:26:44.168283    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eec5852_d33e_40f7_8d1f_43e7d7eead44.slice/crio-da5c5635f18c81f82b3f0b5e23c134497717f36a53b1bcd36c9128bbd05dec29 WatchSource:0}: Error finding container da5c5635f18c81f82b3f0b5e23c134497717f36a53b1bcd36c9128bbd05dec29: Status 404 returned error can't find the container with id da5c5635f18c81f82b3f0b5e23c134497717f36a53b1bcd36c9128bbd05dec29
Jul 27 15:26:44.917528 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:44.917490    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg" event={"ID":"0eec5852-d33e-40f7-8d1f-43e7d7eead44","Type":"ContainerStarted","Data":"c1ce75f053962d42cdfa1d1b60ec7b9f383a6acbd86bfe2b8827adab2638f69d"}
Jul 27 15:26:44.917979 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:44.917562    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg" event={"ID":"0eec5852-d33e-40f7-8d1f-43e7d7eead44","Type":"ContainerStarted","Data":"da5c5635f18c81f82b3f0b5e23c134497717f36a53b1bcd36c9128bbd05dec29"}
Jul 27 15:26:44.917979 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:44.917645    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg"
Jul 27 15:26:44.944756 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:44.944711    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg" podStartSLOduration=1.944697449 podStartE2EDuration="1.944697449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:26:44.94121986 +0000 UTC m=+1276.369895043" watchObservedRunningTime="2026-07-27 15:26:44.944697449 +0000 UTC m=+1276.373372631"
Jul 27 15:26:55.923768 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:26:55.923688    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqpfg"
Jul 27 15:27:26.110128 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:27:26.110096    2548 generic.go:356] "Generic (PLEG): container finished" podID="ed328f85-503e-4671-83e9-b44f78f30568" containerID="1bd9c35781851ecc6513f9251bee7306375517719c199f03dea30586b5d4c03f" exitCode=1
Jul 27 15:27:26.110500 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:27:26.110167    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g" event={"ID":"ed328f85-503e-4671-83e9-b44f78f30568","Type":"ContainerDied","Data":"1bd9c35781851ecc6513f9251bee7306375517719c199f03dea30586b5d4c03f"}
Jul 27 15:27:26.110500 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:27:26.110443    2548 scope.go:117] "RemoveContainer" containerID="1bd9c35781851ecc6513f9251bee7306375517719c199f03dea30586b5d4c03f"
Jul 27 15:27:26.113616 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:27:26.113598    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 27 15:27:27.116103 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:27:27.116070    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g" event={"ID":"ed328f85-503e-4671-83e9-b44f78f30568","Type":"ContainerStarted","Data":"73831c79c662047311632c94daea7a02676576d2b0a5e3f6a3a9d879a5c3f4bb"}
Jul 27 15:27:27.116704 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:27:27.116278    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g"
Jul 27 15:27:38.122958 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:27:38.122925    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2lf8g"
Jul 27 15:28:40.545427 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:28:40.545394    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:28:51.416949 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:28:51.416912    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:29:42.970717 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:29:42.970674    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:29:53.812970 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:29:53.812938    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:30:02.331906 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:30:02.331870    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:30:14.837662 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:30:14.837613    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:30:15.619036 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:30:15.619000    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:33:14.324778 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:33:14.324742    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:33:15.095637 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:33:15.095601    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:33:33.363371 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:33:33.363340    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:33:57.694385 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:33:57.694307    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:00.710973 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:00.710938    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:00.816187 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:00.816154    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:01.181989 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:01.181957    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:06.900946 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:06.900917    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:07.000283 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:07.000248    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:07.133482 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:07.133448    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:07.509993 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:07.509950    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:08.133875 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:08.133839    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:09.742006 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:09.741967    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:09.813252 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:09.813221    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"]
Jul 27 15:34:09.817134 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:09.817111    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:09.819836 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:09.819812    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-654abfac-serving-cert\""
Jul 27 15:34:09.829224 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:09.829199    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"]
Jul 27 15:34:09.835980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:09.835958    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:09.925159 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:09.925129    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6npq\" (UniqueName: \"kubernetes.io/projected/77381e6e-90c6-4ce9-84e0-597726490893-kube-api-access-c6npq\") pod \"maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts\" (UID: \"77381e6e-90c6-4ce9-84e0-597726490893\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:09.925305 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:09.925195    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77381e6e-90c6-4ce9-84e0-597726490893-openshift-service-ca\") pod \"maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts\" (UID: \"77381e6e-90c6-4ce9-84e0-597726490893\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:09.925305 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:09.925244    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77381e6e-90c6-4ce9-84e0-597726490893-maas-api-tls\") pod \"maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts\" (UID: \"77381e6e-90c6-4ce9-84e0-597726490893\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:09.936467 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:09.936440    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:10.025961 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:10.025870    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6npq\" (UniqueName: \"kubernetes.io/projected/77381e6e-90c6-4ce9-84e0-597726490893-kube-api-access-c6npq\") pod \"maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts\" (UID: \"77381e6e-90c6-4ce9-84e0-597726490893\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:10.025961 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:10.025940    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77381e6e-90c6-4ce9-84e0-597726490893-openshift-service-ca\") pod \"maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts\" (UID: \"77381e6e-90c6-4ce9-84e0-597726490893\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:10.026184 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:10.025989    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77381e6e-90c6-4ce9-84e0-597726490893-maas-api-tls\") pod \"maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts\" (UID: \"77381e6e-90c6-4ce9-84e0-597726490893\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:10.026184 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:34:10.026088    2548 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-serving-cert: secret "maas-api-e2e-ait-654abfac-serving-cert" not found
Jul 27 15:34:10.026184 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:34:10.026160    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77381e6e-90c6-4ce9-84e0-597726490893-maas-api-tls podName:77381e6e-90c6-4ce9-84e0-597726490893 nodeName:}" failed. No retries permitted until 2026-07-27 15:34:10.526142112 +0000 UTC m=+1721.954817275 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/77381e6e-90c6-4ce9-84e0-597726490893-maas-api-tls") pod "maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts" (UID: "77381e6e-90c6-4ce9-84e0-597726490893") : secret "maas-api-e2e-ait-654abfac-serving-cert" not found
Jul 27 15:34:10.026592 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:10.026573    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77381e6e-90c6-4ce9-84e0-597726490893-openshift-service-ca\") pod \"maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts\" (UID: \"77381e6e-90c6-4ce9-84e0-597726490893\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:10.035219 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:10.035193    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6npq\" (UniqueName: \"kubernetes.io/projected/77381e6e-90c6-4ce9-84e0-597726490893-kube-api-access-c6npq\") pod \"maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts\" (UID: \"77381e6e-90c6-4ce9-84e0-597726490893\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:10.142750 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:10.142713    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:10.531459 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:10.531427    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77381e6e-90c6-4ce9-84e0-597726490893-maas-api-tls\") pod \"maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts\" (UID: \"77381e6e-90c6-4ce9-84e0-597726490893\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:10.534164 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:10.534137    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77381e6e-90c6-4ce9-84e0-597726490893-maas-api-tls\") pod \"maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts\" (UID: \"77381e6e-90c6-4ce9-84e0-597726490893\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:10.728403 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:10.728368    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:10.872843 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:10.872810    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"]
Jul 27 15:34:10.875254 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:34:10.875228    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77381e6e_90c6_4ce9_84e0_597726490893.slice/crio-c6bccaa7bacfb5a0d4f299331d53e9b8716799be7b488c3c1139f5249f677500 WatchSource:0}: Error finding container c6bccaa7bacfb5a0d4f299331d53e9b8716799be7b488c3c1139f5249f677500: Status 404 returned error can't find the container with id c6bccaa7bacfb5a0d4f299331d53e9b8716799be7b488c3c1139f5249f677500
Jul 27 15:34:10.877047 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:10.877030    2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 15:34:11.043945 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:11.043843    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts" event={"ID":"77381e6e-90c6-4ce9-84e0-597726490893","Type":"ContainerStarted","Data":"c6bccaa7bacfb5a0d4f299331d53e9b8716799be7b488c3c1139f5249f677500"}
Jul 27 15:34:11.118969 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:11.118930    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:11.219455 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:11.219420    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:12.049986 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:12.049954    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts" event={"ID":"77381e6e-90c6-4ce9-84e0-597726490893","Type":"ContainerStarted","Data":"7617c0fcf094ec1a0f264965719ec0dc3bbc236b3066da814b166be6b501b428"}
Jul 27 15:34:12.050335 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:12.050080    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:12.074245 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:12.074194    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts" podStartSLOduration=2.004472654 podStartE2EDuration="3.074178027s" totalImagesPullingTime="1.069705373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:34:12.072869408 +0000 UTC m=+1723.501544585" watchObservedRunningTime="2026-07-27 15:34:12.074178027 +0000 UTC m=+1723.502853216"
Jul 27 15:34:18.059343 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:18.059312    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:45.640780 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:45.640734    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"]
Jul 27 15:34:45.641379 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:45.640988    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts" podUID="77381e6e-90c6-4ce9-84e0-597726490893" containerName="maas-api" containerID="cri-o://7617c0fcf094ec1a0f264965719ec0dc3bbc236b3066da814b166be6b501b428" gracePeriod=30
Jul 27 15:34:45.671935 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:45.671908    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:45.774592 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:45.774548    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:45.870479 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:45.870451    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:34:45.891411 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:45.891350    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:46.035523 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.035469    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77381e6e-90c6-4ce9-84e0-597726490893-maas-api-tls\") pod \"77381e6e-90c6-4ce9-84e0-597726490893\" (UID: \"77381e6e-90c6-4ce9-84e0-597726490893\") "
Jul 27 15:34:46.035745 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.035591    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77381e6e-90c6-4ce9-84e0-597726490893-openshift-service-ca\") pod \"77381e6e-90c6-4ce9-84e0-597726490893\" (UID: \"77381e6e-90c6-4ce9-84e0-597726490893\") "
Jul 27 15:34:46.035745 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.035632    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6npq\" (UniqueName: \"kubernetes.io/projected/77381e6e-90c6-4ce9-84e0-597726490893-kube-api-access-c6npq\") pod \"77381e6e-90c6-4ce9-84e0-597726490893\" (UID: \"77381e6e-90c6-4ce9-84e0-597726490893\") "
Jul 27 15:34:46.035940 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.035917    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77381e6e-90c6-4ce9-84e0-597726490893-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "77381e6e-90c6-4ce9-84e0-597726490893" (UID: "77381e6e-90c6-4ce9-84e0-597726490893"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:34:46.037902 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.037870    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77381e6e-90c6-4ce9-84e0-597726490893-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "77381e6e-90c6-4ce9-84e0-597726490893" (UID: "77381e6e-90c6-4ce9-84e0-597726490893"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:34:46.038016 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.037953    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77381e6e-90c6-4ce9-84e0-597726490893-kube-api-access-c6npq" (OuterVolumeSpecName: "kube-api-access-c6npq") pod "77381e6e-90c6-4ce9-84e0-597726490893" (UID: "77381e6e-90c6-4ce9-84e0-597726490893"). InnerVolumeSpecName "kube-api-access-c6npq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:34:46.136847 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.136817    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77381e6e-90c6-4ce9-84e0-597726490893-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:34:46.136847 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.136841    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77381e6e-90c6-4ce9-84e0-597726490893-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:34:46.136847 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.136852    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6npq\" (UniqueName: \"kubernetes.io/projected/77381e6e-90c6-4ce9-84e0-597726490893-kube-api-access-c6npq\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:34:46.213299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.213209    2548 generic.go:356] "Generic (PLEG): container finished" podID="77381e6e-90c6-4ce9-84e0-597726490893" containerID="7617c0fcf094ec1a0f264965719ec0dc3bbc236b3066da814b166be6b501b428" exitCode=0
Jul 27 15:34:46.213299 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.213260    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"
Jul 27 15:34:46.213519 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.213303    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts" event={"ID":"77381e6e-90c6-4ce9-84e0-597726490893","Type":"ContainerDied","Data":"7617c0fcf094ec1a0f264965719ec0dc3bbc236b3066da814b166be6b501b428"}
Jul 27 15:34:46.213519 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.213358    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts" event={"ID":"77381e6e-90c6-4ce9-84e0-597726490893","Type":"ContainerDied","Data":"c6bccaa7bacfb5a0d4f299331d53e9b8716799be7b488c3c1139f5249f677500"}
Jul 27 15:34:46.213519 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.213384    2548 scope.go:117] "RemoveContainer" containerID="7617c0fcf094ec1a0f264965719ec0dc3bbc236b3066da814b166be6b501b428"
Jul 27 15:34:46.221889 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.221873    2548 scope.go:117] "RemoveContainer" containerID="7617c0fcf094ec1a0f264965719ec0dc3bbc236b3066da814b166be6b501b428"
Jul 27 15:34:46.222142 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:34:46.222124    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7617c0fcf094ec1a0f264965719ec0dc3bbc236b3066da814b166be6b501b428\": container with ID starting with 7617c0fcf094ec1a0f264965719ec0dc3bbc236b3066da814b166be6b501b428 not found: ID does not exist" containerID="7617c0fcf094ec1a0f264965719ec0dc3bbc236b3066da814b166be6b501b428"
Jul 27 15:34:46.222194 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.222150    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7617c0fcf094ec1a0f264965719ec0dc3bbc236b3066da814b166be6b501b428"} err="failed to get container status \"7617c0fcf094ec1a0f264965719ec0dc3bbc236b3066da814b166be6b501b428\": rpc error: code = NotFound desc = could not find container \"7617c0fcf094ec1a0f264965719ec0dc3bbc236b3066da814b166be6b501b428\": container with ID starting with 7617c0fcf094ec1a0f264965719ec0dc3bbc236b3066da814b166be6b501b428 not found: ID does not exist"
Jul 27 15:34:46.233804 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.233779    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"]
Jul 27 15:34:46.242559 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:46.241983    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-654abfac-66fcb8ccd8-zqjts"]
Jul 27 15:34:47.139731 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:47.139700    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77381e6e-90c6-4ce9-84e0-597726490893" path="/var/lib/kubelet/pods/77381e6e-90c6-4ce9-84e0-597726490893/volumes"
Jul 27 15:34:50.364553 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:34:50.364497    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:00.512269 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:00.512235    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:00.818942 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:00.818856    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:01.026821 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:01.026786    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:01.126802 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:01.126702    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:01.226480 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:01.226449    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:01.527745 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:01.527710    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:01.672463 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:01.672430    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:18.128517 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:18.128484    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:18.700128 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:18.700088    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:29.323461 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:29.323425    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:29.953141 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:29.953110    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:38.356027 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.355994    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:38.439174 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.439138    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"]
Jul 27 15:35:38.439558 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.439545    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77381e6e-90c6-4ce9-84e0-597726490893" containerName="maas-api"
Jul 27 15:35:38.439611 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.439559    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="77381e6e-90c6-4ce9-84e0-597726490893" containerName="maas-api"
Jul 27 15:35:38.439647 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.439622    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="77381e6e-90c6-4ce9-84e0-597726490893" containerName="maas-api"
Jul 27 15:35:38.443926 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.443911    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:35:38.446477 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.446455    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-58d49f55-serving-cert\""
Jul 27 15:35:38.453759 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.453735    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"]
Jul 27 15:35:38.551140 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.551083    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:38.593326 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.593292    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlq4m\" (UniqueName: \"kubernetes.io/projected/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-kube-api-access-vlq4m\") pod \"maas-api-e2e-migrate-58d49f55-6f655b449c-hx772\" (UID: \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:35:38.593326 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.593328    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-openshift-service-ca\") pod \"maas-api-e2e-migrate-58d49f55-6f655b449c-hx772\" (UID: \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:35:38.593524 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.593374    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-maas-api-tls\") pod \"maas-api-e2e-migrate-58d49f55-6f655b449c-hx772\" (UID: \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:35:38.694565 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.694468    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-openshift-service-ca\") pod \"maas-api-e2e-migrate-58d49f55-6f655b449c-hx772\" (UID: \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:35:38.694704 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.694582    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-maas-api-tls\") pod \"maas-api-e2e-migrate-58d49f55-6f655b449c-hx772\" (UID: \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:35:38.694704 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.694664    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlq4m\" (UniqueName: \"kubernetes.io/projected/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-kube-api-access-vlq4m\") pod \"maas-api-e2e-migrate-58d49f55-6f655b449c-hx772\" (UID: \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:35:38.694777 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:35:38.694734    2548 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-serving-cert: secret "maas-api-e2e-migrate-58d49f55-serving-cert" not found
Jul 27 15:35:38.694837 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:35:38.694825    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-maas-api-tls podName:50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8 nodeName:}" failed. No retries permitted until 2026-07-27 15:35:39.194802232 +0000 UTC m=+1810.623477395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-maas-api-tls") pod "maas-api-e2e-migrate-58d49f55-6f655b449c-hx772" (UID: "50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8") : secret "maas-api-e2e-migrate-58d49f55-serving-cert" not found
Jul 27 15:35:38.695185 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.695169    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-openshift-service-ca\") pod \"maas-api-e2e-migrate-58d49f55-6f655b449c-hx772\" (UID: \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:35:38.703481 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.703453    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlq4m\" (UniqueName: \"kubernetes.io/projected/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-kube-api-access-vlq4m\") pod \"maas-api-e2e-migrate-58d49f55-6f655b449c-hx772\" (UID: \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:35:38.900883 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:38.900852    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:39.001102 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:39.001045    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:39.114124 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:39.114087    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:39.199690 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:39.199654    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-maas-api-tls\") pod \"maas-api-e2e-migrate-58d49f55-6f655b449c-hx772\" (UID: \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:35:39.202471 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:39.202445    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-maas-api-tls\") pod \"maas-api-e2e-migrate-58d49f55-6f655b449c-hx772\" (UID: \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:35:39.355820 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:39.355728    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:35:39.495038 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:39.495012    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"]
Jul 27 15:35:39.496590 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:35:39.496554    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50c509f5_d7c8_4fc8_b1c2_c50c6793b2d8.slice/crio-efe9a439a139ebd9a27dcb841eb366e258ef9dcc73c790ca8c8c20a3817f41a0 WatchSource:0}: Error finding container efe9a439a139ebd9a27dcb841eb366e258ef9dcc73c790ca8c8c20a3817f41a0: Status 404 returned error can't find the container with id efe9a439a139ebd9a27dcb841eb366e258ef9dcc73c790ca8c8c20a3817f41a0
Jul 27 15:35:40.467707 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:40.467667    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772" event={"ID":"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8","Type":"ContainerStarted","Data":"5cb135b720b8b38a3df1a486dc566b974fbf5100a64ff356f36c92b6c52d7e0c"}
Jul 27 15:35:40.467885 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:40.467727    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772" event={"ID":"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8","Type":"ContainerStarted","Data":"efe9a439a139ebd9a27dcb841eb366e258ef9dcc73c790ca8c8c20a3817f41a0"}
Jul 27 15:35:40.467885 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:40.467757    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:35:40.485395 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:40.485339    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772" podStartSLOduration=2.025867035 podStartE2EDuration="2.485319685s" totalImagesPullingTime="459.45265ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:35:40.484600294 +0000 UTC m=+1811.913275479" watchObservedRunningTime="2026-07-27 15:35:40.485319685 +0000 UTC m=+1811.913994869"
Jul 27 15:35:46.477237 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:46.477207    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:35:58.636634 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:58.636605    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:35:58.698204 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:35:58.698172    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:00.849647 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:00.849614    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:00.909167 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:00.909132    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:03.351227 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:03.351191    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x"]
Jul 27 15:36:03.354862 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:03.354844    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x"
Jul 27 15:36:03.357359 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:03.357339    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bgfwg\""
Jul 27 15:36:03.361238 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:03.361213    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x"]
Jul 27 15:36:03.419613 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:03.419563    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3019ee1-5746-4a87-beec-1153a68c80b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x\" (UID: \"b3019ee1-5746-4a87-beec-1153a68c80b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x"
Jul 27 15:36:03.520881 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:03.520846    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3019ee1-5746-4a87-beec-1153a68c80b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x\" (UID: \"b3019ee1-5746-4a87-beec-1153a68c80b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x"
Jul 27 15:36:03.521481 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:03.521460    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3019ee1-5746-4a87-beec-1153a68c80b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x\" (UID: \"b3019ee1-5746-4a87-beec-1153a68c80b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x"
Jul 27 15:36:03.668278 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:03.668187    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x"
Jul 27 15:36:03.804621 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:03.804595    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x"]
Jul 27 15:36:03.805915 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:36:03.805889    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3019ee1_5746_4a87_beec_1153a68c80b9.slice/crio-7326b1e135d62de92cfd6048825c57b2dbec55c712b569d0e9f9d56b60bbb9cd WatchSource:0}: Error finding container 7326b1e135d62de92cfd6048825c57b2dbec55c712b569d0e9f9d56b60bbb9cd: Status 404 returned error can't find the container with id 7326b1e135d62de92cfd6048825c57b2dbec55c712b569d0e9f9d56b60bbb9cd
Jul 27 15:36:04.581378 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:04.581339    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x" event={"ID":"b3019ee1-5746-4a87-beec-1153a68c80b9","Type":"ContainerStarted","Data":"7326b1e135d62de92cfd6048825c57b2dbec55c712b569d0e9f9d56b60bbb9cd"}
Jul 27 15:36:07.603609 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:07.603573    2548 generic.go:356] "Generic (PLEG): container finished" podID="b3019ee1-5746-4a87-beec-1153a68c80b9" containerID="1777f0dff6d5deeb20dc70a733f2188bf06c74ec891d3db4c271a1e4b70952be" exitCode=0
Jul 27 15:36:07.604116 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:07.603659    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x" event={"ID":"b3019ee1-5746-4a87-beec-1153a68c80b9","Type":"ContainerDied","Data":"1777f0dff6d5deeb20dc70a733f2188bf06c74ec891d3db4c271a1e4b70952be"}
Jul 27 15:36:08.739917 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:08.739893    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x"
Jul 27 15:36:08.871864 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:08.871775    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3019ee1-5746-4a87-beec-1153a68c80b9-maas-api-service-ca\") pod \"b3019ee1-5746-4a87-beec-1153a68c80b9\" (UID: \"b3019ee1-5746-4a87-beec-1153a68c80b9\") "
Jul 27 15:36:08.872192 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:08.872169    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3019ee1-5746-4a87-beec-1153a68c80b9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b3019ee1-5746-4a87-beec-1153a68c80b9" (UID: "b3019ee1-5746-4a87-beec-1153a68c80b9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:36:08.972763 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:08.972728    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3019ee1-5746-4a87-beec-1153a68c80b9-maas-api-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:36:09.614680 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:09.614647    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x" event={"ID":"b3019ee1-5746-4a87-beec-1153a68c80b9","Type":"ContainerDied","Data":"7326b1e135d62de92cfd6048825c57b2dbec55c712b569d0e9f9d56b60bbb9cd"}
Jul 27 15:36:09.614680 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:09.614679    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x"
Jul 27 15:36:09.614898 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:09.614695    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7326b1e135d62de92cfd6048825c57b2dbec55c712b569d0e9f9d56b60bbb9cd"
Jul 27 15:36:13.803139 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:13.803107    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"]
Jul 27 15:36:13.803613 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:13.803313    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772" podUID="50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8" containerName="maas-api" containerID="cri-o://5cb135b720b8b38a3df1a486dc566b974fbf5100a64ff356f36c92b6c52d7e0c" gracePeriod=30
Jul 27 15:36:13.899008 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:13.898975    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:14.053006 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.052980    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:36:14.058849 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.058786    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:14.119759 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.119722    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-maas-api-tls\") pod \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\" (UID: \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\") "
Jul 27 15:36:14.119924 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.119775    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlq4m\" (UniqueName: \"kubernetes.io/projected/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-kube-api-access-vlq4m\") pod \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\" (UID: \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\") "
Jul 27 15:36:14.119995 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.119975    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-openshift-service-ca\") pod \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\" (UID: \"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8\") "
Jul 27 15:36:14.120297 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.120272    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8" (UID: "50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:36:14.122047 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.122019    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-kube-api-access-vlq4m" (OuterVolumeSpecName: "kube-api-access-vlq4m") pod "50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8" (UID: "50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8"). InnerVolumeSpecName "kube-api-access-vlq4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:36:14.122047 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.122030    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8" (UID: "50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:36:14.221018 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.220986    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:36:14.221018 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.221015    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlq4m\" (UniqueName: \"kubernetes.io/projected/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-kube-api-access-vlq4m\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:36:14.221211 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.221033    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:36:14.638894 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.638859    2548 generic.go:356] "Generic (PLEG): container finished" podID="50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8" containerID="5cb135b720b8b38a3df1a486dc566b974fbf5100a64ff356f36c92b6c52d7e0c" exitCode=0
Jul 27 15:36:14.639078 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.638904    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772" event={"ID":"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8","Type":"ContainerDied","Data":"5cb135b720b8b38a3df1a486dc566b974fbf5100a64ff356f36c92b6c52d7e0c"}
Jul 27 15:36:14.639078 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.638958    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772" event={"ID":"50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8","Type":"ContainerDied","Data":"efe9a439a139ebd9a27dcb841eb366e258ef9dcc73c790ca8c8c20a3817f41a0"}
Jul 27 15:36:14.639078 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.638912    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"
Jul 27 15:36:14.639078 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.638990    2548 scope.go:117] "RemoveContainer" containerID="5cb135b720b8b38a3df1a486dc566b974fbf5100a64ff356f36c92b6c52d7e0c"
Jul 27 15:36:14.648480 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.648465    2548 scope.go:117] "RemoveContainer" containerID="5cb135b720b8b38a3df1a486dc566b974fbf5100a64ff356f36c92b6c52d7e0c"
Jul 27 15:36:14.648756 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:36:14.648737    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cb135b720b8b38a3df1a486dc566b974fbf5100a64ff356f36c92b6c52d7e0c\": container with ID starting with 5cb135b720b8b38a3df1a486dc566b974fbf5100a64ff356f36c92b6c52d7e0c not found: ID does not exist" containerID="5cb135b720b8b38a3df1a486dc566b974fbf5100a64ff356f36c92b6c52d7e0c"
Jul 27 15:36:14.648804 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.648765    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cb135b720b8b38a3df1a486dc566b974fbf5100a64ff356f36c92b6c52d7e0c"} err="failed to get container status \"5cb135b720b8b38a3df1a486dc566b974fbf5100a64ff356f36c92b6c52d7e0c\": rpc error: code = NotFound desc = could not find container \"5cb135b720b8b38a3df1a486dc566b974fbf5100a64ff356f36c92b6c52d7e0c\": container with ID starting with 5cb135b720b8b38a3df1a486dc566b974fbf5100a64ff356f36c92b6c52d7e0c not found: ID does not exist"
Jul 27 15:36:14.659832 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.659809    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"]
Jul 27 15:36:14.663646 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:14.663626    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-58d49f55-6f655b449c-hx772"]
Jul 27 15:36:15.138920 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:15.138886    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8" path="/var/lib/kubelet/pods/50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8/volumes"
Jul 27 15:36:18.434364 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:18.434324    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x"]
Jul 27 15:36:18.437416 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:18.437388    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-58d49f55-2tt6x"]
Jul 27 15:36:18.438705 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:18.438658    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:18.582919 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:18.582878    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:19.139060 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:19.139019    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3019ee1-5746-4a87-beec-1153a68c80b9" path="/var/lib/kubelet/pods/b3019ee1-5746-4a87-beec-1153a68c80b9/volumes"
Jul 27 15:36:28.840593 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:28.840560    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:29.087588 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:29.087524    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:29.446268 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:29.446229    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:30.291032 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:30.290996    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:31.551424 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.551390    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:31.597192 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.597159    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"]
Jul 27 15:36:31.597597 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.597582    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3019ee1-5746-4a87-beec-1153a68c80b9" containerName="revoke-keys"
Jul 27 15:36:31.597648 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.597599    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3019ee1-5746-4a87-beec-1153a68c80b9" containerName="revoke-keys"
Jul 27 15:36:31.597648 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.597615    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8" containerName="maas-api"
Jul 27 15:36:31.597648 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.597620    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8" containerName="maas-api"
Jul 27 15:36:31.597740 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.597686    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3019ee1-5746-4a87-beec-1153a68c80b9" containerName="revoke-keys"
Jul 27 15:36:31.597740 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.597695    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="50c509f5-d7c8-4fc8-b1c2-c50c6793b2d8" containerName="maas-api"
Jul 27 15:36:31.601955 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.601939    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:36:31.604583 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.604561    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-3af40866-serving-cert\""
Jul 27 15:36:31.613962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.613940    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"]
Jul 27 15:36:31.674664 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.674632    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ef51af1-f623-4388-a2bb-7e4f66985c87-maas-api-tls\") pod \"maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp\" (UID: \"9ef51af1-f623-4388-a2bb-7e4f66985c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:36:31.674851 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.674688    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xplvj\" (UniqueName: \"kubernetes.io/projected/9ef51af1-f623-4388-a2bb-7e4f66985c87-kube-api-access-xplvj\") pod \"maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp\" (UID: \"9ef51af1-f623-4388-a2bb-7e4f66985c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:36:31.674851 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.674774    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef51af1-f623-4388-a2bb-7e4f66985c87-openshift-service-ca\") pod \"maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp\" (UID: \"9ef51af1-f623-4388-a2bb-7e4f66985c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:36:31.688229 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.688190    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:31.775919 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.775886    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef51af1-f623-4388-a2bb-7e4f66985c87-openshift-service-ca\") pod \"maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp\" (UID: \"9ef51af1-f623-4388-a2bb-7e4f66985c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:36:31.776080 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.775962    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ef51af1-f623-4388-a2bb-7e4f66985c87-maas-api-tls\") pod \"maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp\" (UID: \"9ef51af1-f623-4388-a2bb-7e4f66985c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:36:31.776080 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.775989    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xplvj\" (UniqueName: \"kubernetes.io/projected/9ef51af1-f623-4388-a2bb-7e4f66985c87-kube-api-access-xplvj\") pod \"maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp\" (UID: \"9ef51af1-f623-4388-a2bb-7e4f66985c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:36:31.776171 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:36:31.776102    2548 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-serving-cert: secret "maas-api-e2e-ait-3af40866-serving-cert" not found
Jul 27 15:36:31.776210 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:36:31.776183    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ef51af1-f623-4388-a2bb-7e4f66985c87-maas-api-tls podName:9ef51af1-f623-4388-a2bb-7e4f66985c87 nodeName:}" failed. No retries permitted until 2026-07-27 15:36:32.276161179 +0000 UTC m=+1863.704836346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9ef51af1-f623-4388-a2bb-7e4f66985c87-maas-api-tls") pod "maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp" (UID: "9ef51af1-f623-4388-a2bb-7e4f66985c87") : secret "maas-api-e2e-ait-3af40866-serving-cert" not found
Jul 27 15:36:31.776649 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.776633    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef51af1-f623-4388-a2bb-7e4f66985c87-openshift-service-ca\") pod \"maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp\" (UID: \"9ef51af1-f623-4388-a2bb-7e4f66985c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:36:31.789776 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.789751    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xplvj\" (UniqueName: \"kubernetes.io/projected/9ef51af1-f623-4388-a2bb-7e4f66985c87-kube-api-access-xplvj\") pod \"maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp\" (UID: \"9ef51af1-f623-4388-a2bb-7e4f66985c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:36:31.976793 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:31.976767    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:32.280675 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.280559    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ef51af1-f623-4388-a2bb-7e4f66985c87-maas-api-tls\") pod \"maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp\" (UID: \"9ef51af1-f623-4388-a2bb-7e4f66985c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:36:32.283165 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.283144    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ef51af1-f623-4388-a2bb-7e4f66985c87-maas-api-tls\") pod \"maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp\" (UID: \"9ef51af1-f623-4388-a2bb-7e4f66985c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:36:32.414916 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.414878    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:32.512231 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.512198    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:36:32.514252 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.514228    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:36:32.604566 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.604395    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"]
Jul 27 15:36:32.610355 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.609496    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"
Jul 27 15:36:32.612810 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.612702    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-3af40866-dockercfg-8gn82\""
Jul 27 15:36:32.613984 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.613869    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-3af40866\""
Jul 27 15:36:32.618091 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.618027    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"]
Jul 27 15:36:32.663813 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.663753    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"]
Jul 27 15:36:32.664900 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:36:32.664874    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef51af1_f623_4388_a2bb_7e4f66985c87.slice/crio-2abe72712c60048ba2c04605b41583b7b1c69010ba3a76b61aae7894421a34f7 WatchSource:0}: Error finding container 2abe72712c60048ba2c04605b41583b7b1c69010ba3a76b61aae7894421a34f7: Status 404 returned error can't find the container with id 2abe72712c60048ba2c04605b41583b7b1c69010ba3a76b61aae7894421a34f7
Jul 27 15:36:32.685324 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.685295    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fe0ab84-3fb1-4703-98b3-e1bdfdaad395-plugins-config-volume\") pod \"payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6\" (UID: \"4fe0ab84-3fb1-4703-98b3-e1bdfdaad395\") " pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"
Jul 27 15:36:32.685485 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.685379    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlqbm\" (UniqueName: \"kubernetes.io/projected/4fe0ab84-3fb1-4703-98b3-e1bdfdaad395-kube-api-access-hlqbm\") pod \"payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6\" (UID: \"4fe0ab84-3fb1-4703-98b3-e1bdfdaad395\") " pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"
Jul 27 15:36:32.726916 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.726869    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp" event={"ID":"9ef51af1-f623-4388-a2bb-7e4f66985c87","Type":"ContainerStarted","Data":"2abe72712c60048ba2c04605b41583b7b1c69010ba3a76b61aae7894421a34f7"}
Jul 27 15:36:32.786959 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.786920    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlqbm\" (UniqueName: \"kubernetes.io/projected/4fe0ab84-3fb1-4703-98b3-e1bdfdaad395-kube-api-access-hlqbm\") pod \"payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6\" (UID: \"4fe0ab84-3fb1-4703-98b3-e1bdfdaad395\") " pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"
Jul 27 15:36:32.787163 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.787062    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fe0ab84-3fb1-4703-98b3-e1bdfdaad395-plugins-config-volume\") pod \"payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6\" (UID: \"4fe0ab84-3fb1-4703-98b3-e1bdfdaad395\") " pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"
Jul 27 15:36:32.787973 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.787941    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fe0ab84-3fb1-4703-98b3-e1bdfdaad395-plugins-config-volume\") pod \"payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6\" (UID: \"4fe0ab84-3fb1-4703-98b3-e1bdfdaad395\") " pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"
Jul 27 15:36:32.796586 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.796563    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlqbm\" (UniqueName: \"kubernetes.io/projected/4fe0ab84-3fb1-4703-98b3-e1bdfdaad395-kube-api-access-hlqbm\") pod \"payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6\" (UID: \"4fe0ab84-3fb1-4703-98b3-e1bdfdaad395\") " pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"
Jul 27 15:36:32.925608 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:32.925485    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"
Jul 27 15:36:33.070894 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:33.070725    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"]
Jul 27 15:36:33.073052 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:36:33.073019    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fe0ab84_3fb1_4703_98b3_e1bdfdaad395.slice/crio-6f8f4bd166c027bac857de1ef8f69cb61c6790522f72e1f0cd25c948ed056d33 WatchSource:0}: Error finding container 6f8f4bd166c027bac857de1ef8f69cb61c6790522f72e1f0cd25c948ed056d33: Status 404 returned error can't find the container with id 6f8f4bd166c027bac857de1ef8f69cb61c6790522f72e1f0cd25c948ed056d33
Jul 27 15:36:33.733520 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:33.733486    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6" event={"ID":"4fe0ab84-3fb1-4703-98b3-e1bdfdaad395","Type":"ContainerStarted","Data":"682c6122960cf3e668fb563878537eeb9a383f723674dc2e0f70a041c262b3af"}
Jul 27 15:36:33.733941 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:33.733556    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6" event={"ID":"4fe0ab84-3fb1-4703-98b3-e1bdfdaad395","Type":"ContainerStarted","Data":"6f8f4bd166c027bac857de1ef8f69cb61c6790522f72e1f0cd25c948ed056d33"}
Jul 27 15:36:33.733941 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:33.733587    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"
Jul 27 15:36:33.756281 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:33.756230    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6" podStartSLOduration=1.756215025 podStartE2EDuration="1.756215025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:36:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:36:33.754852811 +0000 UTC m=+1865.183527993" watchObservedRunningTime="2026-07-27 15:36:33.756215025 +0000 UTC m=+1865.184890254"
Jul 27 15:36:34.740592 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:34.740546    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp" event={"ID":"9ef51af1-f623-4388-a2bb-7e4f66985c87","Type":"ContainerStarted","Data":"65ec9d189fb020e0387655ca2c77a4410717ec241c0db6d5db162c6357f6bc26"}
Jul 27 15:36:34.763899 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:34.763847    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp" podStartSLOduration=2.295873678 podStartE2EDuration="3.763832612s" totalImagesPullingTime="1.467958934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:36:34.7627069 +0000 UTC m=+1866.191382083" watchObservedRunningTime="2026-07-27 15:36:34.763832612 +0000 UTC m=+1866.192508231"
Jul 27 15:36:35.746064 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:35.746032    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:36:41.755894 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:41.755862    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:36:44.742860 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:44.742832    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"
Jul 27 15:36:57.322749 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:57.322664    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m"]
Jul 27 15:36:57.329760 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:57.329735    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m"
Jul 27 15:36:57.332577 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:57.332555    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bgfwg\""
Jul 27 15:36:57.336580 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:57.336560    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m"]
Jul 27 15:36:57.414099 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:57.414069    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17688d9a-85dd-4b9d-9619-6cf8818d30ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3af40866-svd7m\" (UID: \"17688d9a-85dd-4b9d-9619-6cf8818d30ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m"
Jul 27 15:36:57.515294 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:57.515264    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17688d9a-85dd-4b9d-9619-6cf8818d30ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3af40866-svd7m\" (UID: \"17688d9a-85dd-4b9d-9619-6cf8818d30ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m"
Jul 27 15:36:57.515923 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:57.515907    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17688d9a-85dd-4b9d-9619-6cf8818d30ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3af40866-svd7m\" (UID: \"17688d9a-85dd-4b9d-9619-6cf8818d30ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m"
Jul 27 15:36:57.641555 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:57.641475    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m"
Jul 27 15:36:57.775329 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:57.775275    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m"]
Jul 27 15:36:57.776299 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:36:57.776268    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17688d9a_85dd_4b9d_9619_6cf8818d30ac.slice/crio-7e9043b788070af4a2069fddf0713cb29b6a0d2d0d9feccbfe2f3950b44b064b WatchSource:0}: Error finding container 7e9043b788070af4a2069fddf0713cb29b6a0d2d0d9feccbfe2f3950b44b064b: Status 404 returned error can't find the container with id 7e9043b788070af4a2069fddf0713cb29b6a0d2d0d9feccbfe2f3950b44b064b
Jul 27 15:36:57.855745 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:57.855719    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m" event={"ID":"17688d9a-85dd-4b9d-9619-6cf8818d30ac","Type":"ContainerStarted","Data":"7e9043b788070af4a2069fddf0713cb29b6a0d2d0d9feccbfe2f3950b44b064b"}
Jul 27 15:36:58.863551 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:58.863442    2548 generic.go:356] "Generic (PLEG): container finished" podID="17688d9a-85dd-4b9d-9619-6cf8818d30ac" containerID="97b2ec9f057e8bd3a3db830ebaa9ea9ac88a1d1d41090c4cea05d6d4e65f87fb" exitCode=0
Jul 27 15:36:58.863935 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:36:58.863546    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m" event={"ID":"17688d9a-85dd-4b9d-9619-6cf8818d30ac","Type":"ContainerDied","Data":"97b2ec9f057e8bd3a3db830ebaa9ea9ac88a1d1d41090c4cea05d6d4e65f87fb"}
Jul 27 15:37:00.015511 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:00.015484    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m"
Jul 27 15:37:00.139944 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:00.139863    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17688d9a-85dd-4b9d-9619-6cf8818d30ac-maas-api-service-ca\") pod \"17688d9a-85dd-4b9d-9619-6cf8818d30ac\" (UID: \"17688d9a-85dd-4b9d-9619-6cf8818d30ac\") "
Jul 27 15:37:00.140227 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:00.140208    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17688d9a-85dd-4b9d-9619-6cf8818d30ac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "17688d9a-85dd-4b9d-9619-6cf8818d30ac" (UID: "17688d9a-85dd-4b9d-9619-6cf8818d30ac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:37:00.241156 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:00.241125    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17688d9a-85dd-4b9d-9619-6cf8818d30ac-maas-api-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:37:00.873890 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:00.873858    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m" event={"ID":"17688d9a-85dd-4b9d-9619-6cf8818d30ac","Type":"ContainerDied","Data":"7e9043b788070af4a2069fddf0713cb29b6a0d2d0d9feccbfe2f3950b44b064b"}
Jul 27 15:37:00.873890 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:00.873885    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m"
Jul 27 15:37:00.874189 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:00.873906    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e9043b788070af4a2069fddf0713cb29b6a0d2d0d9feccbfe2f3950b44b064b"
Jul 27 15:37:07.851926 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:07.851884    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"]
Jul 27 15:37:07.852360 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:07.852133    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp" podUID="9ef51af1-f623-4388-a2bb-7e4f66985c87" containerName="maas-api" containerID="cri-o://65ec9d189fb020e0387655ca2c77a4410717ec241c0db6d5db162c6357f6bc26" gracePeriod=30
Jul 27 15:37:07.881766 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:07.881734    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:07.905178 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:07.905147    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"]
Jul 27 15:37:07.905828 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:07.905718    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6" podUID="4fe0ab84-3fb1-4703-98b3-e1bdfdaad395" containerName="payload-processing" containerID="cri-o://682c6122960cf3e668fb563878537eeb9a383f723674dc2e0f70a041c262b3af" gracePeriod=30
Jul 27 15:37:08.037855 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.037804    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:08.138914 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.138840    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:08.154139 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.154117    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:37:08.181811 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.181789    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"
Jul 27 15:37:08.314357 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.314327    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef51af1-f623-4388-a2bb-7e4f66985c87-openshift-service-ca\") pod \"9ef51af1-f623-4388-a2bb-7e4f66985c87\" (UID: \"9ef51af1-f623-4388-a2bb-7e4f66985c87\") "
Jul 27 15:37:08.314520 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.314385    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ef51af1-f623-4388-a2bb-7e4f66985c87-maas-api-tls\") pod \"9ef51af1-f623-4388-a2bb-7e4f66985c87\" (UID: \"9ef51af1-f623-4388-a2bb-7e4f66985c87\") "
Jul 27 15:37:08.314520 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.314445    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlqbm\" (UniqueName: \"kubernetes.io/projected/4fe0ab84-3fb1-4703-98b3-e1bdfdaad395-kube-api-access-hlqbm\") pod \"4fe0ab84-3fb1-4703-98b3-e1bdfdaad395\" (UID: \"4fe0ab84-3fb1-4703-98b3-e1bdfdaad395\") "
Jul 27 15:37:08.314520 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.314464    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xplvj\" (UniqueName: \"kubernetes.io/projected/9ef51af1-f623-4388-a2bb-7e4f66985c87-kube-api-access-xplvj\") pod \"9ef51af1-f623-4388-a2bb-7e4f66985c87\" (UID: \"9ef51af1-f623-4388-a2bb-7e4f66985c87\") "
Jul 27 15:37:08.314520 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.314500    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fe0ab84-3fb1-4703-98b3-e1bdfdaad395-plugins-config-volume\") pod \"4fe0ab84-3fb1-4703-98b3-e1bdfdaad395\" (UID: \"4fe0ab84-3fb1-4703-98b3-e1bdfdaad395\") "
Jul 27 15:37:08.315085 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.314959    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fe0ab84-3fb1-4703-98b3-e1bdfdaad395-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4fe0ab84-3fb1-4703-98b3-e1bdfdaad395" (UID: "4fe0ab84-3fb1-4703-98b3-e1bdfdaad395"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:37:08.315085 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.314979    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ef51af1-f623-4388-a2bb-7e4f66985c87-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ef51af1-f623-4388-a2bb-7e4f66985c87" (UID: "9ef51af1-f623-4388-a2bb-7e4f66985c87"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:37:08.316721 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.316698    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fe0ab84-3fb1-4703-98b3-e1bdfdaad395-kube-api-access-hlqbm" (OuterVolumeSpecName: "kube-api-access-hlqbm") pod "4fe0ab84-3fb1-4703-98b3-e1bdfdaad395" (UID: "4fe0ab84-3fb1-4703-98b3-e1bdfdaad395"). InnerVolumeSpecName "kube-api-access-hlqbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:37:08.316815 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.316740    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ef51af1-f623-4388-a2bb-7e4f66985c87-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ef51af1-f623-4388-a2bb-7e4f66985c87" (UID: "9ef51af1-f623-4388-a2bb-7e4f66985c87"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:37:08.316919 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.316898    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ef51af1-f623-4388-a2bb-7e4f66985c87-kube-api-access-xplvj" (OuterVolumeSpecName: "kube-api-access-xplvj") pod "9ef51af1-f623-4388-a2bb-7e4f66985c87" (UID: "9ef51af1-f623-4388-a2bb-7e4f66985c87"). InnerVolumeSpecName "kube-api-access-xplvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:37:08.416202 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.416125    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef51af1-f623-4388-a2bb-7e4f66985c87-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:37:08.416202 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.416151    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ef51af1-f623-4388-a2bb-7e4f66985c87-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:37:08.416202 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.416160    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlqbm\" (UniqueName: \"kubernetes.io/projected/4fe0ab84-3fb1-4703-98b3-e1bdfdaad395-kube-api-access-hlqbm\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:37:08.416202 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.416168    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xplvj\" (UniqueName: \"kubernetes.io/projected/9ef51af1-f623-4388-a2bb-7e4f66985c87-kube-api-access-xplvj\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:37:08.416202 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.416191    2548 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fe0ab84-3fb1-4703-98b3-e1bdfdaad395-plugins-config-volume\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:37:08.918435 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.918402    2548 generic.go:356] "Generic (PLEG): container finished" podID="4fe0ab84-3fb1-4703-98b3-e1bdfdaad395" containerID="682c6122960cf3e668fb563878537eeb9a383f723674dc2e0f70a041c262b3af" exitCode=0
Jul 27 15:37:08.918864 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.918462    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"
Jul 27 15:37:08.918864 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.918494    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6" event={"ID":"4fe0ab84-3fb1-4703-98b3-e1bdfdaad395","Type":"ContainerDied","Data":"682c6122960cf3e668fb563878537eeb9a383f723674dc2e0f70a041c262b3af"}
Jul 27 15:37:08.918864 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.918602    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6" event={"ID":"4fe0ab84-3fb1-4703-98b3-e1bdfdaad395","Type":"ContainerDied","Data":"6f8f4bd166c027bac857de1ef8f69cb61c6790522f72e1f0cd25c948ed056d33"}
Jul 27 15:37:08.918864 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.918633    2548 scope.go:117] "RemoveContainer" containerID="682c6122960cf3e668fb563878537eeb9a383f723674dc2e0f70a041c262b3af"
Jul 27 15:37:08.919856 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.919834    2548 generic.go:356] "Generic (PLEG): container finished" podID="9ef51af1-f623-4388-a2bb-7e4f66985c87" containerID="65ec9d189fb020e0387655ca2c77a4410717ec241c0db6d5db162c6357f6bc26" exitCode=0
Jul 27 15:37:08.919923 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.919905    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"
Jul 27 15:37:08.919980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.919917    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp" event={"ID":"9ef51af1-f623-4388-a2bb-7e4f66985c87","Type":"ContainerDied","Data":"65ec9d189fb020e0387655ca2c77a4410717ec241c0db6d5db162c6357f6bc26"}
Jul 27 15:37:08.919980 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.919969    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp" event={"ID":"9ef51af1-f623-4388-a2bb-7e4f66985c87","Type":"ContainerDied","Data":"2abe72712c60048ba2c04605b41583b7b1c69010ba3a76b61aae7894421a34f7"}
Jul 27 15:37:08.927689 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.927674    2548 scope.go:117] "RemoveContainer" containerID="682c6122960cf3e668fb563878537eeb9a383f723674dc2e0f70a041c262b3af"
Jul 27 15:37:08.927980 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:37:08.927953    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"682c6122960cf3e668fb563878537eeb9a383f723674dc2e0f70a041c262b3af\": container with ID starting with 682c6122960cf3e668fb563878537eeb9a383f723674dc2e0f70a041c262b3af not found: ID does not exist" containerID="682c6122960cf3e668fb563878537eeb9a383f723674dc2e0f70a041c262b3af"
Jul 27 15:37:08.928072 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.927993    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"682c6122960cf3e668fb563878537eeb9a383f723674dc2e0f70a041c262b3af"} err="failed to get container status \"682c6122960cf3e668fb563878537eeb9a383f723674dc2e0f70a041c262b3af\": rpc error: code = NotFound desc = could not find container \"682c6122960cf3e668fb563878537eeb9a383f723674dc2e0f70a041c262b3af\": container with ID starting with 682c6122960cf3e668fb563878537eeb9a383f723674dc2e0f70a041c262b3af not found: ID does not exist"
Jul 27 15:37:08.928072 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.928017    2548 scope.go:117] "RemoveContainer" containerID="65ec9d189fb020e0387655ca2c77a4410717ec241c0db6d5db162c6357f6bc26"
Jul 27 15:37:08.936245 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.936224    2548 scope.go:117] "RemoveContainer" containerID="65ec9d189fb020e0387655ca2c77a4410717ec241c0db6d5db162c6357f6bc26"
Jul 27 15:37:08.936469 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:37:08.936452    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65ec9d189fb020e0387655ca2c77a4410717ec241c0db6d5db162c6357f6bc26\": container with ID starting with 65ec9d189fb020e0387655ca2c77a4410717ec241c0db6d5db162c6357f6bc26 not found: ID does not exist" containerID="65ec9d189fb020e0387655ca2c77a4410717ec241c0db6d5db162c6357f6bc26"
Jul 27 15:37:08.936528 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.936475    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65ec9d189fb020e0387655ca2c77a4410717ec241c0db6d5db162c6357f6bc26"} err="failed to get container status \"65ec9d189fb020e0387655ca2c77a4410717ec241c0db6d5db162c6357f6bc26\": rpc error: code = NotFound desc = could not find container \"65ec9d189fb020e0387655ca2c77a4410717ec241c0db6d5db162c6357f6bc26\": container with ID starting with 65ec9d189fb020e0387655ca2c77a4410717ec241c0db6d5db162c6357f6bc26 not found: ID does not exist"
Jul 27 15:37:08.945856 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.945835    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"]
Jul 27 15:37:08.949594 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.949575    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3af40866-8bfd689dd-pdrfp"]
Jul 27 15:37:08.961032 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.961008    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"]
Jul 27 15:37:08.964529 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:08.964509    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3af40866-747858cbb4-lv2l6"]
Jul 27 15:37:09.139586 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:09.139488    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fe0ab84-3fb1-4703-98b3-e1bdfdaad395" path="/var/lib/kubelet/pods/4fe0ab84-3fb1-4703-98b3-e1bdfdaad395/volumes"
Jul 27 15:37:09.139977 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:09.139956    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ef51af1-f623-4388-a2bb-7e4f66985c87" path="/var/lib/kubelet/pods/9ef51af1-f623-4388-a2bb-7e4f66985c87/volumes"
Jul 27 15:37:12.379763 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:12.379731    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m"]
Jul 27 15:37:12.381717 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:12.381689    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3af40866-svd7m"]
Jul 27 15:37:12.382209 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:12.382171    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:13.139172 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:13.139136    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17688d9a-85dd-4b9d-9619-6cf8818d30ac" path="/var/lib/kubelet/pods/17688d9a-85dd-4b9d-9619-6cf8818d30ac/volumes"
Jul 27 15:37:13.613361 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:13.613332    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:13.826404 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:13.826367    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:14.059275 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:14.059236    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:14.347929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:14.347842    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:14.785487 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:14.785461    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:16.709971 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.709918    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:16.800034 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.800003    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"]
Jul 27 15:37:16.800418 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.800403    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fe0ab84-3fb1-4703-98b3-e1bdfdaad395" containerName="payload-processing"
Jul 27 15:37:16.800462 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.800422    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fe0ab84-3fb1-4703-98b3-e1bdfdaad395" containerName="payload-processing"
Jul 27 15:37:16.800462 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.800432    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ef51af1-f623-4388-a2bb-7e4f66985c87" containerName="maas-api"
Jul 27 15:37:16.800462 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.800438    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ef51af1-f623-4388-a2bb-7e4f66985c87" containerName="maas-api"
Jul 27 15:37:16.800462 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.800452    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17688d9a-85dd-4b9d-9619-6cf8818d30ac" containerName="revoke-keys"
Jul 27 15:37:16.800462 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.800458    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="17688d9a-85dd-4b9d-9619-6cf8818d30ac" containerName="revoke-keys"
Jul 27 15:37:16.800638 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.800523    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ef51af1-f623-4388-a2bb-7e4f66985c87" containerName="maas-api"
Jul 27 15:37:16.800638 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.800544    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fe0ab84-3fb1-4703-98b3-e1bdfdaad395" containerName="payload-processing"
Jul 27 15:37:16.800638 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.800550    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="17688d9a-85dd-4b9d-9619-6cf8818d30ac" containerName="revoke-keys"
Jul 27 15:37:16.804866 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.804847    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:16.807454 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.807436    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-b2ba96cb-serving-cert\""
Jul 27 15:37:16.816077 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.816056    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"]
Jul 27 15:37:16.833249 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.833231    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:16.891744 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.891715    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkvhm\" (UniqueName: \"kubernetes.io/projected/d7c6a638-74aa-43ab-805c-f1835de52218-kube-api-access-tkvhm\") pod \"maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8\" (UID: \"d7c6a638-74aa-43ab-805c-f1835de52218\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:16.892006 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.891798    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c6a638-74aa-43ab-805c-f1835de52218-openshift-service-ca\") pod \"maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8\" (UID: \"d7c6a638-74aa-43ab-805c-f1835de52218\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:16.892006 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.891870    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7c6a638-74aa-43ab-805c-f1835de52218-maas-api-tls\") pod \"maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8\" (UID: \"d7c6a638-74aa-43ab-805c-f1835de52218\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:16.935106 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.935079    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:16.992444 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.992410    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkvhm\" (UniqueName: \"kubernetes.io/projected/d7c6a638-74aa-43ab-805c-f1835de52218-kube-api-access-tkvhm\") pod \"maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8\" (UID: \"d7c6a638-74aa-43ab-805c-f1835de52218\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:16.992629 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.992461    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c6a638-74aa-43ab-805c-f1835de52218-openshift-service-ca\") pod \"maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8\" (UID: \"d7c6a638-74aa-43ab-805c-f1835de52218\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:16.992629 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.992503    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7c6a638-74aa-43ab-805c-f1835de52218-maas-api-tls\") pod \"maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8\" (UID: \"d7c6a638-74aa-43ab-805c-f1835de52218\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:16.992768 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:37:16.992674    2548 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-serving-cert: secret "maas-api-e2e-derive-b2ba96cb-serving-cert" not found
Jul 27 15:37:16.992768 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:37:16.992752    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7c6a638-74aa-43ab-805c-f1835de52218-maas-api-tls podName:d7c6a638-74aa-43ab-805c-f1835de52218 nodeName:}" failed. No retries permitted until 2026-07-27 15:37:17.492730657 +0000 UTC m=+1908.921405826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d7c6a638-74aa-43ab-805c-f1835de52218-maas-api-tls") pod "maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8" (UID: "d7c6a638-74aa-43ab-805c-f1835de52218") : secret "maas-api-e2e-derive-b2ba96cb-serving-cert" not found
Jul 27 15:37:16.993118 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:16.993100    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c6a638-74aa-43ab-805c-f1835de52218-openshift-service-ca\") pod \"maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8\" (UID: \"d7c6a638-74aa-43ab-805c-f1835de52218\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:17.008879 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.008857    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkvhm\" (UniqueName: \"kubernetes.io/projected/d7c6a638-74aa-43ab-805c-f1835de52218-kube-api-access-tkvhm\") pod \"maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8\" (UID: \"d7c6a638-74aa-43ab-805c-f1835de52218\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:17.310199 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.310116    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:17.439039 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.439007    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:17.496517 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.496482    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7c6a638-74aa-43ab-805c-f1835de52218-maas-api-tls\") pod \"maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8\" (UID: \"d7c6a638-74aa-43ab-805c-f1835de52218\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:17.499101 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.499078    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7c6a638-74aa-43ab-805c-f1835de52218-maas-api-tls\") pod \"maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8\" (UID: \"d7c6a638-74aa-43ab-805c-f1835de52218\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:17.572103 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.572019    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:17.695803 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.695772    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"]
Jul 27 15:37:17.699508 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.699487    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"
Jul 27 15:37:17.702024 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.702007    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-b2ba96cb\""
Jul 27 15:37:17.702117 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.702005    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-b2ba96cb-dockercfg-ftphz\""
Jul 27 15:37:17.715975 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.715953    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"]
Jul 27 15:37:17.716319 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.716029    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:17.800193 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.800080    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpn6f\" (UniqueName: \"kubernetes.io/projected/dcf4b1bd-0dd6-483c-a75e-744485ec68de-kube-api-access-bpn6f\") pod \"payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j\" (UID: \"dcf4b1bd-0dd6-483c-a75e-744485ec68de\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"
Jul 27 15:37:17.800193 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.800183    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcf4b1bd-0dd6-483c-a75e-744485ec68de-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j\" (UID: \"dcf4b1bd-0dd6-483c-a75e-744485ec68de\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"
Jul 27 15:37:17.859271 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.859237    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"]
Jul 27 15:37:17.861349 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:37:17.861325    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7c6a638_74aa_43ab_805c_f1835de52218.slice/crio-74a76fcacd019fc038b86cd5d1faad4da07588889b0534f69f515203c2f732a4 WatchSource:0}: Error finding container 74a76fcacd019fc038b86cd5d1faad4da07588889b0534f69f515203c2f732a4: Status 404 returned error can't find the container with id 74a76fcacd019fc038b86cd5d1faad4da07588889b0534f69f515203c2f732a4
Jul 27 15:37:17.900752 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.900721    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpn6f\" (UniqueName: \"kubernetes.io/projected/dcf4b1bd-0dd6-483c-a75e-744485ec68de-kube-api-access-bpn6f\") pod \"payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j\" (UID: \"dcf4b1bd-0dd6-483c-a75e-744485ec68de\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"
Jul 27 15:37:17.900900 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.900771    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcf4b1bd-0dd6-483c-a75e-744485ec68de-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j\" (UID: \"dcf4b1bd-0dd6-483c-a75e-744485ec68de\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"
Jul 27 15:37:17.901397 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.901379    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcf4b1bd-0dd6-483c-a75e-744485ec68de-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j\" (UID: \"dcf4b1bd-0dd6-483c-a75e-744485ec68de\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"
Jul 27 15:37:17.910384 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.910364    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpn6f\" (UniqueName: \"kubernetes.io/projected/dcf4b1bd-0dd6-483c-a75e-744485ec68de-kube-api-access-bpn6f\") pod \"payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j\" (UID: \"dcf4b1bd-0dd6-483c-a75e-744485ec68de\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"
Jul 27 15:37:17.965242 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:17.965212    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8" event={"ID":"d7c6a638-74aa-43ab-805c-f1835de52218","Type":"ContainerStarted","Data":"74a76fcacd019fc038b86cd5d1faad4da07588889b0534f69f515203c2f732a4"}
Jul 27 15:37:18.012588 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:18.012555    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"
Jul 27 15:37:18.147682 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:18.147653    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"]
Jul 27 15:37:18.149344 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:37:18.149314    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf4b1bd_0dd6_483c_a75e_744485ec68de.slice/crio-75f10464ba749d1f1677929100a49b7388b1ef9421fa27d3f1e2ae9b5b2df22a WatchSource:0}: Error finding container 75f10464ba749d1f1677929100a49b7388b1ef9421fa27d3f1e2ae9b5b2df22a: Status 404 returned error can't find the container with id 75f10464ba749d1f1677929100a49b7388b1ef9421fa27d3f1e2ae9b5b2df22a
Jul 27 15:37:18.971592 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:18.971560    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j" event={"ID":"dcf4b1bd-0dd6-483c-a75e-744485ec68de","Type":"ContainerStarted","Data":"b9987923954a98e595e926d864f45903922d7bb3b3a1e1dad8893302ebff136b"}
Jul 27 15:37:18.971948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:18.971653    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j" event={"ID":"dcf4b1bd-0dd6-483c-a75e-744485ec68de","Type":"ContainerStarted","Data":"75f10464ba749d1f1677929100a49b7388b1ef9421fa27d3f1e2ae9b5b2df22a"}
Jul 27 15:37:18.971948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:18.971679    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"
Jul 27 15:37:18.972981 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:18.972953    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8" event={"ID":"d7c6a638-74aa-43ab-805c-f1835de52218","Type":"ContainerStarted","Data":"a74274a3130a390065ecfba575b326d9669db27b4e823d5164132e0656b912a2"}
Jul 27 15:37:18.973074 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:18.973001    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:18.990060 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:18.990012    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j" podStartSLOduration=1.9899948109999999 podStartE2EDuration="1.989994811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:37:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:37:18.987452477 +0000 UTC m=+1910.416127664" watchObservedRunningTime="2026-07-27 15:37:18.989994811 +0000 UTC m=+1910.418669999"
Jul 27 15:37:19.007193 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:19.007151    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8" podStartSLOduration=1.969034204 podStartE2EDuration="3.007138392s" totalImagesPullingTime="1.038104188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:37:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:37:19.006426777 +0000 UTC m=+1910.435101960" watchObservedRunningTime="2026-07-27 15:37:19.007138392 +0000 UTC m=+1910.435813611"
Jul 27 15:37:24.984045 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:24.984017    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:29.980585 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:29.980522    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"
Jul 27 15:37:51.971443 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:51.971405    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"]
Jul 27 15:37:51.972404 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:51.971778    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8" podUID="d7c6a638-74aa-43ab-805c-f1835de52218" containerName="maas-api" containerID="cri-o://a74274a3130a390065ecfba575b326d9669db27b4e823d5164132e0656b912a2" gracePeriod=30
Jul 27 15:37:51.997991 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:51.997966    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:52.027378 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.027349    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"]
Jul 27 15:37:52.027633 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.027593    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j" podUID="dcf4b1bd-0dd6-483c-a75e-744485ec68de" containerName="payload-pre-processing" containerID="cri-o://b9987923954a98e595e926d864f45903922d7bb3b3a1e1dad8893302ebff136b" gracePeriod=30
Jul 27 15:37:52.098273 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.098246    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:52.138400 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.138370    2548 generic.go:356] "Generic (PLEG): container finished" podID="d7c6a638-74aa-43ab-805c-f1835de52218" containerID="a74274a3130a390065ecfba575b326d9669db27b4e823d5164132e0656b912a2" exitCode=0
Jul 27 15:37:52.138523 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.138401    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8" event={"ID":"d7c6a638-74aa-43ab-805c-f1835de52218","Type":"ContainerDied","Data":"a74274a3130a390065ecfba575b326d9669db27b4e823d5164132e0656b912a2"}
Jul 27 15:37:52.277360 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.277340    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:52.288193 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.288171    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"
Jul 27 15:37:52.417908 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.417869    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c6a638-74aa-43ab-805c-f1835de52218-openshift-service-ca\") pod \"d7c6a638-74aa-43ab-805c-f1835de52218\" (UID: \"d7c6a638-74aa-43ab-805c-f1835de52218\") "
Jul 27 15:37:52.418093 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.417944    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7c6a638-74aa-43ab-805c-f1835de52218-maas-api-tls\") pod \"d7c6a638-74aa-43ab-805c-f1835de52218\" (UID: \"d7c6a638-74aa-43ab-805c-f1835de52218\") "
Jul 27 15:37:52.418093 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.417967    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpn6f\" (UniqueName: \"kubernetes.io/projected/dcf4b1bd-0dd6-483c-a75e-744485ec68de-kube-api-access-bpn6f\") pod \"dcf4b1bd-0dd6-483c-a75e-744485ec68de\" (UID: \"dcf4b1bd-0dd6-483c-a75e-744485ec68de\") "
Jul 27 15:37:52.418093 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.418010    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkvhm\" (UniqueName: \"kubernetes.io/projected/d7c6a638-74aa-43ab-805c-f1835de52218-kube-api-access-tkvhm\") pod \"d7c6a638-74aa-43ab-805c-f1835de52218\" (UID: \"d7c6a638-74aa-43ab-805c-f1835de52218\") "
Jul 27 15:37:52.418255 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.418094    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcf4b1bd-0dd6-483c-a75e-744485ec68de-plugins-config-volume\") pod \"dcf4b1bd-0dd6-483c-a75e-744485ec68de\" (UID: \"dcf4b1bd-0dd6-483c-a75e-744485ec68de\") "
Jul 27 15:37:52.418316 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.418246    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7c6a638-74aa-43ab-805c-f1835de52218-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d7c6a638-74aa-43ab-805c-f1835de52218" (UID: "d7c6a638-74aa-43ab-805c-f1835de52218"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:37:52.418520 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.418493    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcf4b1bd-0dd6-483c-a75e-744485ec68de-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dcf4b1bd-0dd6-483c-a75e-744485ec68de" (UID: "dcf4b1bd-0dd6-483c-a75e-744485ec68de"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:37:52.418669 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.418521    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c6a638-74aa-43ab-805c-f1835de52218-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:37:52.420224 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.420197    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7c6a638-74aa-43ab-805c-f1835de52218-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d7c6a638-74aa-43ab-805c-f1835de52218" (UID: "d7c6a638-74aa-43ab-805c-f1835de52218"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:37:52.420323 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.420236    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcf4b1bd-0dd6-483c-a75e-744485ec68de-kube-api-access-bpn6f" (OuterVolumeSpecName: "kube-api-access-bpn6f") pod "dcf4b1bd-0dd6-483c-a75e-744485ec68de" (UID: "dcf4b1bd-0dd6-483c-a75e-744485ec68de"). InnerVolumeSpecName "kube-api-access-bpn6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:37:52.420323 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.420242    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7c6a638-74aa-43ab-805c-f1835de52218-kube-api-access-tkvhm" (OuterVolumeSpecName: "kube-api-access-tkvhm") pod "d7c6a638-74aa-43ab-805c-f1835de52218" (UID: "d7c6a638-74aa-43ab-805c-f1835de52218"). InnerVolumeSpecName "kube-api-access-tkvhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:37:52.519868 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.519789    2548 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcf4b1bd-0dd6-483c-a75e-744485ec68de-plugins-config-volume\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:37:52.519868 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.519814    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7c6a638-74aa-43ab-805c-f1835de52218-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:37:52.519868 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.519823    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpn6f\" (UniqueName: \"kubernetes.io/projected/dcf4b1bd-0dd6-483c-a75e-744485ec68de-kube-api-access-bpn6f\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:37:52.519868 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.519832    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkvhm\" (UniqueName: \"kubernetes.io/projected/d7c6a638-74aa-43ab-805c-f1835de52218-kube-api-access-tkvhm\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:37:52.531962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:52.531939    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:53.146578 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:53.146528    2548 generic.go:356] "Generic (PLEG): container finished" podID="dcf4b1bd-0dd6-483c-a75e-744485ec68de" containerID="b9987923954a98e595e926d864f45903922d7bb3b3a1e1dad8893302ebff136b" exitCode=0
Jul 27 15:37:53.147570 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:53.146608    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"
Jul 27 15:37:53.147570 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:53.146634    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j" event={"ID":"dcf4b1bd-0dd6-483c-a75e-744485ec68de","Type":"ContainerDied","Data":"b9987923954a98e595e926d864f45903922d7bb3b3a1e1dad8893302ebff136b"}
Jul 27 15:37:53.147570 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:53.146688    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j" event={"ID":"dcf4b1bd-0dd6-483c-a75e-744485ec68de","Type":"ContainerDied","Data":"75f10464ba749d1f1677929100a49b7388b1ef9421fa27d3f1e2ae9b5b2df22a"}
Jul 27 15:37:53.147570 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:53.146716    2548 scope.go:117] "RemoveContainer" containerID="b9987923954a98e595e926d864f45903922d7bb3b3a1e1dad8893302ebff136b"
Jul 27 15:37:53.148921 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:53.148627    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8" event={"ID":"d7c6a638-74aa-43ab-805c-f1835de52218","Type":"ContainerDied","Data":"74a76fcacd019fc038b86cd5d1faad4da07588889b0534f69f515203c2f732a4"}
Jul 27 15:37:53.148921 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:53.148641    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"
Jul 27 15:37:53.158648 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:53.158632    2548 scope.go:117] "RemoveContainer" containerID="b9987923954a98e595e926d864f45903922d7bb3b3a1e1dad8893302ebff136b"
Jul 27 15:37:53.158909 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:37:53.158889    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9987923954a98e595e926d864f45903922d7bb3b3a1e1dad8893302ebff136b\": container with ID starting with b9987923954a98e595e926d864f45903922d7bb3b3a1e1dad8893302ebff136b not found: ID does not exist" containerID="b9987923954a98e595e926d864f45903922d7bb3b3a1e1dad8893302ebff136b"
Jul 27 15:37:53.158970 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:53.158917    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9987923954a98e595e926d864f45903922d7bb3b3a1e1dad8893302ebff136b"} err="failed to get container status \"b9987923954a98e595e926d864f45903922d7bb3b3a1e1dad8893302ebff136b\": rpc error: code = NotFound desc = could not find container \"b9987923954a98e595e926d864f45903922d7bb3b3a1e1dad8893302ebff136b\": container with ID starting with b9987923954a98e595e926d864f45903922d7bb3b3a1e1dad8893302ebff136b not found: ID does not exist"
Jul 27 15:37:53.158970 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:53.158934    2548 scope.go:117] "RemoveContainer" containerID="a74274a3130a390065ecfba575b326d9669db27b4e823d5164132e0656b912a2"
Jul 27 15:37:53.170060 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:53.170040    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"]
Jul 27 15:37:53.173611 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:53.173591    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b2ba96cb-7bd55b68f6-4bml8"]
Jul 27 15:37:53.186584 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:53.186563    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"]
Jul 27 15:37:53.190444 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:53.190425    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b2ba96cb-6547b75d4d-4g54j"]
Jul 27 15:37:55.138794 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:55.138761    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7c6a638-74aa-43ab-805c-f1835de52218" path="/var/lib/kubelet/pods/d7c6a638-74aa-43ab-805c-f1835de52218/volumes"
Jul 27 15:37:55.139165 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:55.139122    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcf4b1bd-0dd6-483c-a75e-744485ec68de" path="/var/lib/kubelet/pods/dcf4b1bd-0dd6-483c-a75e-744485ec68de/volumes"
Jul 27 15:37:56.623838 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:56.623803    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:56.779753 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:56.779724    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:57.021870 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:57.021834    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:57.483235 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:57.483204    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:57.572655 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:57.572626    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:57.672557 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:57.672508    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:37:59.111936 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:37:59.111895    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:38:04.837692 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:38:04.837645    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:38:04.896792 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:38:04.896762    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:38:04.960757 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:38:04.960724    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:38:05.061862 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:38:05.061833    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:38:05.161904 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:38:05.161823    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:38:05.559346 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:38:05.559317    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:38:31.548764 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:38:31.548710    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:38:37.959147 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:38:37.959105    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:38:38.058032 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:38:38.057996    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:38:38.159323 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:38:38.159284    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:38:54.054021 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:38:54.053976    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:38:54.566259 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:38:54.566224    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:38:59.727005 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:38:59.726971    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:02.597198 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:02.597164    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:10.595130 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:10.595098    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:11.412964 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:11.412934    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:18.098551 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:18.098505    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:18.198182 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:18.198135    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:18.298990 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:18.298957    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:18.400705 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:18.400618    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:19.157452 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:19.157416    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:19.258197 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:19.258158    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:19.356398 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:19.356360    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:21.117416 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:21.117382    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:27.065130 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:27.065101    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:27.897915 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:27.897882    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:27.961931 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:27.961895    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:33.741305 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:33.741270    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:33.811493 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:33.811463    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:33.975239 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:33.975208    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:34.268771 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:34.268734    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:34.625269 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:34.625179    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:36.963912 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:36.963873    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:37.063410 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.063380    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:37.159591 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.159558    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"]
Jul 27 15:39:37.160114 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.160085    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcf4b1bd-0dd6-483c-a75e-744485ec68de" containerName="payload-pre-processing"
Jul 27 15:39:37.160114 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.160107    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcf4b1bd-0dd6-483c-a75e-744485ec68de" containerName="payload-pre-processing"
Jul 27 15:39:37.160265 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.160137    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7c6a638-74aa-43ab-805c-f1835de52218" containerName="maas-api"
Jul 27 15:39:37.160265 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.160143    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7c6a638-74aa-43ab-805c-f1835de52218" containerName="maas-api"
Jul 27 15:39:37.160265 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.160204    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7c6a638-74aa-43ab-805c-f1835de52218" containerName="maas-api"
Jul 27 15:39:37.160265 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.160218    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcf4b1bd-0dd6-483c-a75e-744485ec68de" containerName="payload-pre-processing"
Jul 27 15:39:37.163117 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.163103    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:37.165727 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.165707    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-22f7c4a2-serving-cert\""
Jul 27 15:39:37.172791 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.172761    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"]
Jul 27 15:39:37.191419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.191393    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:37.234794 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.234760    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt7ct\" (UniqueName: \"kubernetes.io/projected/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-kube-api-access-bt7ct\") pod \"maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg\" (UID: \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:37.234940 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.234807    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-maas-api-tls\") pod \"maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg\" (UID: \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:37.234940 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.234870    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-openshift-service-ca\") pod \"maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg\" (UID: \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:37.335726 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.335694    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt7ct\" (UniqueName: \"kubernetes.io/projected/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-kube-api-access-bt7ct\") pod \"maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg\" (UID: \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:37.335911 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.335739    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-maas-api-tls\") pod \"maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg\" (UID: \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:37.335911 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.335774    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-openshift-service-ca\") pod \"maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg\" (UID: \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:37.335911 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:39:37.335892    2548 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-serving-cert: secret "maas-api-e2e-mt-22f7c4a2-serving-cert" not found
Jul 27 15:39:37.336101 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:39:37.335961    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-maas-api-tls podName:ec5a877c-61bb-41f4-80e9-eb06e1b0bca0 nodeName:}" failed. No retries permitted until 2026-07-27 15:39:37.83594535 +0000 UTC m=+2049.264620511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-maas-api-tls") pod "maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg" (UID: "ec5a877c-61bb-41f4-80e9-eb06e1b0bca0") : secret "maas-api-e2e-mt-22f7c4a2-serving-cert" not found
Jul 27 15:39:37.336375 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.336358    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-openshift-service-ca\") pod \"maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg\" (UID: \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:37.347419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.347394    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt7ct\" (UniqueName: \"kubernetes.io/projected/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-kube-api-access-bt7ct\") pod \"maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg\" (UID: \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:37.654998 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.654913    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:37.755954 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.755919    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:37.839994 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.839955    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-maas-api-tls\") pod \"maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg\" (UID: \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:37.842478 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:37.842457    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-maas-api-tls\") pod \"maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg\" (UID: \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:38.074669 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:38.074633    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:38.215713 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:38.215683    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"]
Jul 27 15:39:38.217848 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:39:38.217823    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec5a877c_61bb_41f4_80e9_eb06e1b0bca0.slice/crio-0deed45b54c833dd435d1ea87eb7e867e23e76e1843ff294854db399fc6f299e WatchSource:0}: Error finding container 0deed45b54c833dd435d1ea87eb7e867e23e76e1843ff294854db399fc6f299e: Status 404 returned error can't find the container with id 0deed45b54c833dd435d1ea87eb7e867e23e76e1843ff294854db399fc6f299e
Jul 27 15:39:38.219199 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:38.219182    2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 15:39:38.653139 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:38.653105    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg" event={"ID":"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0","Type":"ContainerStarted","Data":"0deed45b54c833dd435d1ea87eb7e867e23e76e1843ff294854db399fc6f299e"}
Jul 27 15:39:39.659650 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:39.659611    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg" event={"ID":"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0","Type":"ContainerStarted","Data":"9c6b747d83e442628f05d1cd14ce07065f3ae186654a4fd3f5c50ca275fe74b5"}
Jul 27 15:39:39.660041 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:39.659677    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:39.676025 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:39.675947    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg" podStartSLOduration=2.144632579 podStartE2EDuration="2.675932853s" totalImagesPullingTime="531.300274ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:39:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:39:39.674950554 +0000 UTC m=+2051.103625736" watchObservedRunningTime="2026-07-27 15:39:39.675932853 +0000 UTC m=+2051.104608088"
Jul 27 15:39:41.450473 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:41.450441    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:42.119370 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:42.119342    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:42.221423 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:42.221389    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:42.564705 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:42.564667    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:45.669813 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:45.669769    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:48.355370 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:48.355333    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:48.455639 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:48.455604    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:48.709674 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:48.709600    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm"]
Jul 27 15:39:48.712894 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:48.712879    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm"
Jul 27 15:39:48.715694 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:48.715665    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bgfwg\""
Jul 27 15:39:48.721162 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:48.721137    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm"]
Jul 27 15:39:48.742642 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:48.742610    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fd6540-80e5-4290-84d2-13732ed3cf19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm\" (UID: \"15fd6540-80e5-4290-84d2-13732ed3cf19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm"
Jul 27 15:39:48.843215 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:48.843180    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fd6540-80e5-4290-84d2-13732ed3cf19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm\" (UID: \"15fd6540-80e5-4290-84d2-13732ed3cf19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm"
Jul 27 15:39:48.843850 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:48.843827    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fd6540-80e5-4290-84d2-13732ed3cf19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm\" (UID: \"15fd6540-80e5-4290-84d2-13732ed3cf19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm"
Jul 27 15:39:49.025483 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:49.025409    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm"
Jul 27 15:39:49.159659 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:49.159632    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm"]
Jul 27 15:39:49.161239 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:39:49.161208    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15fd6540_80e5_4290_84d2_13732ed3cf19.slice/crio-076d62d8bc5330d97a49e591e4446259c18bc918ade42f72f4eabded2f5234e4 WatchSource:0}: Error finding container 076d62d8bc5330d97a49e591e4446259c18bc918ade42f72f4eabded2f5234e4: Status 404 returned error can't find the container with id 076d62d8bc5330d97a49e591e4446259c18bc918ade42f72f4eabded2f5234e4
Jul 27 15:39:49.710511 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:49.710397    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm" event={"ID":"15fd6540-80e5-4290-84d2-13732ed3cf19","Type":"ContainerStarted","Data":"b3eb6ae9c1c02349273f7f380b83a1884b77909fc7aa27a8e2bfc9cef9973e34"}
Jul 27 15:39:49.710511 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:49.710483    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm" event={"ID":"15fd6540-80e5-4290-84d2-13732ed3cf19","Type":"ContainerStarted","Data":"076d62d8bc5330d97a49e591e4446259c18bc918ade42f72f4eabded2f5234e4"}
Jul 27 15:39:49.726197 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:49.726155    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm" podStartSLOduration=1.726130071 podStartE2EDuration="1.726130071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:39:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:39:49.725164088 +0000 UTC m=+2061.153839273" watchObservedRunningTime="2026-07-27 15:39:49.726130071 +0000 UTC m=+2061.154805254"
Jul 27 15:39:50.716552 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:50.716496    2548 generic.go:356] "Generic (PLEG): container finished" podID="15fd6540-80e5-4290-84d2-13732ed3cf19" containerID="b3eb6ae9c1c02349273f7f380b83a1884b77909fc7aa27a8e2bfc9cef9973e34" exitCode=0
Jul 27 15:39:50.716932 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:50.716565    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm" event={"ID":"15fd6540-80e5-4290-84d2-13732ed3cf19","Type":"ContainerDied","Data":"b3eb6ae9c1c02349273f7f380b83a1884b77909fc7aa27a8e2bfc9cef9973e34"}
Jul 27 15:39:51.850194 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:51.850164    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm"
Jul 27 15:39:51.973241 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:51.973156    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fd6540-80e5-4290-84d2-13732ed3cf19-maas-api-service-ca\") pod \"15fd6540-80e5-4290-84d2-13732ed3cf19\" (UID: \"15fd6540-80e5-4290-84d2-13732ed3cf19\") "
Jul 27 15:39:51.973500 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:51.973475    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15fd6540-80e5-4290-84d2-13732ed3cf19-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "15fd6540-80e5-4290-84d2-13732ed3cf19" (UID: "15fd6540-80e5-4290-84d2-13732ed3cf19"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:39:52.074020 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:52.073987    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fd6540-80e5-4290-84d2-13732ed3cf19-maas-api-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:39:52.727949 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:52.727921    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm"
Jul 27 15:39:52.728131 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:52.727954    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm" event={"ID":"15fd6540-80e5-4290-84d2-13732ed3cf19","Type":"ContainerDied","Data":"076d62d8bc5330d97a49e591e4446259c18bc918ade42f72f4eabded2f5234e4"}
Jul 27 15:39:52.728131 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:52.728002    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="076d62d8bc5330d97a49e591e4446259c18bc918ade42f72f4eabded2f5234e4"
Jul 27 15:39:55.177776 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:55.177740    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:55.240266 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:55.240238    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:59.210650 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.210613    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"]
Jul 27 15:39:59.211085 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.210853    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg" podUID="ec5a877c-61bb-41f4-80e9-eb06e1b0bca0" containerName="maas-api" containerID="cri-o://9c6b747d83e442628f05d1cd14ce07065f3ae186654a4fd3f5c50ca275fe74b5" gracePeriod=30
Jul 27 15:39:59.244159 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.244130    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:59.343898 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.343873    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:39:59.454152 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.454108    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:59.537060 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.536993    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-maas-api-tls\") pod \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\" (UID: \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\") "
Jul 27 15:39:59.537231 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.537152    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bt7ct\" (UniqueName: \"kubernetes.io/projected/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-kube-api-access-bt7ct\") pod \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\" (UID: \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\") "
Jul 27 15:39:59.537231 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.537196    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-openshift-service-ca\") pod \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\" (UID: \"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0\") "
Jul 27 15:39:59.537615 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.537583    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ec5a877c-61bb-41f4-80e9-eb06e1b0bca0" (UID: "ec5a877c-61bb-41f4-80e9-eb06e1b0bca0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:39:59.539175 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.539149    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ec5a877c-61bb-41f4-80e9-eb06e1b0bca0" (UID: "ec5a877c-61bb-41f4-80e9-eb06e1b0bca0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:39:59.539282 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.539259    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-kube-api-access-bt7ct" (OuterVolumeSpecName: "kube-api-access-bt7ct") pod "ec5a877c-61bb-41f4-80e9-eb06e1b0bca0" (UID: "ec5a877c-61bb-41f4-80e9-eb06e1b0bca0"). InnerVolumeSpecName "kube-api-access-bt7ct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:39:59.638033 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.638002    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bt7ct\" (UniqueName: \"kubernetes.io/projected/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-kube-api-access-bt7ct\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:39:59.638033 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.638025    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:39:59.638033 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.638035    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:39:59.762998 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.762965    2548 generic.go:356] "Generic (PLEG): container finished" podID="ec5a877c-61bb-41f4-80e9-eb06e1b0bca0" containerID="9c6b747d83e442628f05d1cd14ce07065f3ae186654a4fd3f5c50ca275fe74b5" exitCode=0
Jul 27 15:39:59.762998 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.762998    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg" event={"ID":"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0","Type":"ContainerDied","Data":"9c6b747d83e442628f05d1cd14ce07065f3ae186654a4fd3f5c50ca275fe74b5"}
Jul 27 15:39:59.763197 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.763021    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"
Jul 27 15:39:59.763197 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.763034    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg" event={"ID":"ec5a877c-61bb-41f4-80e9-eb06e1b0bca0","Type":"ContainerDied","Data":"0deed45b54c833dd435d1ea87eb7e867e23e76e1843ff294854db399fc6f299e"}
Jul 27 15:39:59.763197 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.763055    2548 scope.go:117] "RemoveContainer" containerID="9c6b747d83e442628f05d1cd14ce07065f3ae186654a4fd3f5c50ca275fe74b5"
Jul 27 15:39:59.771881 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.771864    2548 scope.go:117] "RemoveContainer" containerID="9c6b747d83e442628f05d1cd14ce07065f3ae186654a4fd3f5c50ca275fe74b5"
Jul 27 15:39:59.772135 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:39:59.772114    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c6b747d83e442628f05d1cd14ce07065f3ae186654a4fd3f5c50ca275fe74b5\": container with ID starting with 9c6b747d83e442628f05d1cd14ce07065f3ae186654a4fd3f5c50ca275fe74b5 not found: ID does not exist" containerID="9c6b747d83e442628f05d1cd14ce07065f3ae186654a4fd3f5c50ca275fe74b5"
Jul 27 15:39:59.772232 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.772141    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c6b747d83e442628f05d1cd14ce07065f3ae186654a4fd3f5c50ca275fe74b5"} err="failed to get container status \"9c6b747d83e442628f05d1cd14ce07065f3ae186654a4fd3f5c50ca275fe74b5\": rpc error: code = NotFound desc = could not find container \"9c6b747d83e442628f05d1cd14ce07065f3ae186654a4fd3f5c50ca275fe74b5\": container with ID starting with 9c6b747d83e442628f05d1cd14ce07065f3ae186654a4fd3f5c50ca275fe74b5 not found: ID does not exist"
Jul 27 15:39:59.785664 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.785639    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"]
Jul 27 15:39:59.788773 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:39:59.788726    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-22f7c4a2-9d4d9686-v6zkg"]
Jul 27 15:40:01.138839 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:01.138807    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec5a877c-61bb-41f4-80e9-eb06e1b0bca0" path="/var/lib/kubelet/pods/ec5a877c-61bb-41f4-80e9-eb06e1b0bca0/volumes"
Jul 27 15:40:03.758983 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:03.758949    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:03.792679 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:03.792650    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm"]
Jul 27 15:40:03.798286 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:03.798263    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22f7c4a2-6qhcm"]
Jul 27 15:40:03.825713 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:03.825684    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:04.315432 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:04.315405    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:05.139092 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:05.139056    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15fd6540-80e5-4290-84d2-13732ed3cf19" path="/var/lib/kubelet/pods/15fd6540-80e5-4290-84d2-13732ed3cf19/volumes"
Jul 27 15:40:10.379633 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:10.379584    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:10.606292 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:10.606230    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:10.687075 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:10.686989    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:10.792261 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:10.792218    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:10.926221 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:10.926188    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:11.041877 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:11.041827    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:11.293548 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:11.293425    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:17.378738 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:17.378693    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:17.484361 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:17.484326    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:17.585559 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:17.585498    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:17.742690 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:17.742658    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:23.714763 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:23.714729    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:23.788420 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:23.788386    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:30.522458 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:30.522423    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:30.853283 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:30.853203    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:30.988976 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:30.988943    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:34.327545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.327491    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:34.400171 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.400140    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"]
Jul 27 15:40:34.400601 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.400584    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15fd6540-80e5-4290-84d2-13732ed3cf19" containerName="revoke-keys"
Jul 27 15:40:34.400703 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.400604    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="15fd6540-80e5-4290-84d2-13732ed3cf19" containerName="revoke-keys"
Jul 27 15:40:34.400703 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.400638    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec5a877c-61bb-41f4-80e9-eb06e1b0bca0" containerName="maas-api"
Jul 27 15:40:34.400703 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.400646    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec5a877c-61bb-41f4-80e9-eb06e1b0bca0" containerName="maas-api"
Jul 27 15:40:34.400855 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.400739    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="15fd6540-80e5-4290-84d2-13732ed3cf19" containerName="revoke-keys"
Jul 27 15:40:34.400855 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.400753    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec5a877c-61bb-41f4-80e9-eb06e1b0bca0" containerName="maas-api"
Jul 27 15:40:34.403939 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.403919    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:40:34.406811 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.406792    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-44b2dff2-serving-cert\""
Jul 27 15:40:34.420587 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.420564    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"]
Jul 27 15:40:34.462816 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.462786    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-openshift-service-ca\") pod \"maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw\" (UID: \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:40:34.462964 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.462832    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-maas-api-tls\") pod \"maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw\" (UID: \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:40:34.462964 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.462921    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq97g\" (UniqueName: \"kubernetes.io/projected/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-kube-api-access-hq97g\") pod \"maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw\" (UID: \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:40:34.488607 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.488581    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:34.563941 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.563913    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-openshift-service-ca\") pod \"maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw\" (UID: \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:40:34.564092 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.563952    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-maas-api-tls\") pod \"maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw\" (UID: \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:40:34.564092 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.563993    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq97g\" (UniqueName: \"kubernetes.io/projected/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-kube-api-access-hq97g\") pod \"maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw\" (UID: \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:40:34.564170 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:40:34.564123    2548 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-serving-cert: secret "maas-api-e2e-mt-44b2dff2-serving-cert" not found
Jul 27 15:40:34.564226 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:40:34.564213    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-maas-api-tls podName:8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e nodeName:}" failed. No retries permitted until 2026-07-27 15:40:35.064189447 +0000 UTC m=+2106.492864607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-maas-api-tls") pod "maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw" (UID: "8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e") : secret "maas-api-e2e-mt-44b2dff2-serving-cert" not found
Jul 27 15:40:34.564599 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.564584    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-openshift-service-ca\") pod \"maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw\" (UID: \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:40:34.575272 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.575248    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq97g\" (UniqueName: \"kubernetes.io/projected/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-kube-api-access-hq97g\") pod \"maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw\" (UID: \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:40:34.609884 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.609812    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:34.807818 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:34.807785    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:35.069101 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:35.069055    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-maas-api-tls\") pod \"maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw\" (UID: \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:40:35.071297 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:35.071280    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-maas-api-tls\") pod \"maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw\" (UID: \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:40:35.217943 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:35.217908    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:35.303355 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:35.303322    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:35.314983 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:35.314958    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:40:35.464416 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:35.464391    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"]
Jul 27 15:40:35.466813 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:40:35.466785    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa6f30f_d8e7_4d6e_b0ab_7618a8e2466e.slice/crio-de9f7b21b19827edfac8e6aa5127b0c8a5c80801bffc6fa48dd42217b5d9e42c WatchSource:0}: Error finding container de9f7b21b19827edfac8e6aa5127b0c8a5c80801bffc6fa48dd42217b5d9e42c: Status 404 returned error can't find the container with id de9f7b21b19827edfac8e6aa5127b0c8a5c80801bffc6fa48dd42217b5d9e42c
Jul 27 15:40:35.944940 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:35.944904    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw" event={"ID":"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e","Type":"ContainerStarted","Data":"de9f7b21b19827edfac8e6aa5127b0c8a5c80801bffc6fa48dd42217b5d9e42c"}
Jul 27 15:40:36.951119 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:36.951084    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw" event={"ID":"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e","Type":"ContainerStarted","Data":"0c850c71d58ed258d23ad5be3c0b5e93b25d51ea735dd958e12088a4534a6a90"}
Jul 27 15:40:36.951501 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:36.951162    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:40:36.972035 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:36.971983    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw" podStartSLOduration=2.427943854 podStartE2EDuration="2.971966272s" totalImagesPullingTime="544.022418ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:40:36.970699728 +0000 UTC m=+2108.399374911" watchObservedRunningTime="2026-07-27 15:40:36.971966272 +0000 UTC m=+2108.400641453"
Jul 27 15:40:38.821443 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:38.821404    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:40:42.961417 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:42.961389    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:40:49.524616 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:40:49.524583    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:41:13.318037 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:13.317997    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d"]
Jul 27 15:41:13.321239 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:13.321219    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d"
Jul 27 15:41:13.324487 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:13.324466    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bgfwg\""
Jul 27 15:41:13.329950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:13.329926    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d"]
Jul 27 15:41:13.421043 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:13.420992    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e5601c5-21de-43cc-b319-32efd8669cd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d\" (UID: \"7e5601c5-21de-43cc-b319-32efd8669cd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d"
Jul 27 15:41:13.521928 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:13.521892    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e5601c5-21de-43cc-b319-32efd8669cd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d\" (UID: \"7e5601c5-21de-43cc-b319-32efd8669cd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d"
Jul 27 15:41:13.522522 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:13.522504    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e5601c5-21de-43cc-b319-32efd8669cd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d\" (UID: \"7e5601c5-21de-43cc-b319-32efd8669cd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d"
Jul 27 15:41:13.632721 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:13.632650    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d"
Jul 27 15:41:13.750328 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:13.750300    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:41:13.766590 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:13.766522    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d"]
Jul 27 15:41:13.767794 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:41:13.767766    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e5601c5_21de_43cc_b319_32efd8669cd9.slice/crio-31cfeb58de97dafa4503979f90e76a15a00ee06a2a2b9b778d701c445d45f806 WatchSource:0}: Error finding container 31cfeb58de97dafa4503979f90e76a15a00ee06a2a2b9b778d701c445d45f806: Status 404 returned error can't find the container with id 31cfeb58de97dafa4503979f90e76a15a00ee06a2a2b9b778d701c445d45f806
Jul 27 15:41:14.130849 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:14.130815    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d" event={"ID":"7e5601c5-21de-43cc-b319-32efd8669cd9","Type":"ContainerStarted","Data":"92bbfb8d352a2ea20492f509e083a4e9fba36a447c317a9c4d28c7931283b2b4"}
Jul 27 15:41:14.130982 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:14.130878    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d" event={"ID":"7e5601c5-21de-43cc-b319-32efd8669cd9","Type":"ContainerStarted","Data":"31cfeb58de97dafa4503979f90e76a15a00ee06a2a2b9b778d701c445d45f806"}
Jul 27 15:41:14.149403 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:14.149357    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d" podStartSLOduration=1.149343968 podStartE2EDuration="1.149343968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:41:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:41:14.147354062 +0000 UTC m=+2145.576029258" watchObservedRunningTime="2026-07-27 15:41:14.149343968 +0000 UTC m=+2145.578019151"
Jul 27 15:41:15.137934 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:15.137899    2548 generic.go:356] "Generic (PLEG): container finished" podID="7e5601c5-21de-43cc-b319-32efd8669cd9" containerID="92bbfb8d352a2ea20492f509e083a4e9fba36a447c317a9c4d28c7931283b2b4" exitCode=0
Jul 27 15:41:15.138392 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:15.138370    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d" event={"ID":"7e5601c5-21de-43cc-b319-32efd8669cd9","Type":"ContainerDied","Data":"92bbfb8d352a2ea20492f509e083a4e9fba36a447c317a9c4d28c7931283b2b4"}
Jul 27 15:41:16.271097 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:16.271072    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d"
Jul 27 15:41:16.349240 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:16.349214    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e5601c5-21de-43cc-b319-32efd8669cd9-maas-api-service-ca\") pod \"7e5601c5-21de-43cc-b319-32efd8669cd9\" (UID: \"7e5601c5-21de-43cc-b319-32efd8669cd9\") "
Jul 27 15:41:16.349526 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:16.349506    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e5601c5-21de-43cc-b319-32efd8669cd9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7e5601c5-21de-43cc-b319-32efd8669cd9" (UID: "7e5601c5-21de-43cc-b319-32efd8669cd9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:41:16.450045 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:16.449973    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e5601c5-21de-43cc-b319-32efd8669cd9-maas-api-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:41:17.148661 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:17.148634    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d"
Jul 27 15:41:17.148809 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:17.148639    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d" event={"ID":"7e5601c5-21de-43cc-b319-32efd8669cd9","Type":"ContainerDied","Data":"31cfeb58de97dafa4503979f90e76a15a00ee06a2a2b9b778d701c445d45f806"}
Jul 27 15:41:17.148809 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:17.148761    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31cfeb58de97dafa4503979f90e76a15a00ee06a2a2b9b778d701c445d45f806"
Jul 27 15:41:29.012648 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.012565    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"]
Jul 27 15:41:29.013209 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.012821    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw" podUID="8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e" containerName="maas-api" containerID="cri-o://0c850c71d58ed258d23ad5be3c0b5e93b25d51ea735dd958e12088a4534a6a90" gracePeriod=30
Jul 27 15:41:29.039569 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.039526    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:41:29.140480 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.140452    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:41:29.215288 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.215246    2548 generic.go:356] "Generic (PLEG): container finished" podID="8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e" containerID="0c850c71d58ed258d23ad5be3c0b5e93b25d51ea735dd958e12088a4534a6a90" exitCode=0
Jul 27 15:41:29.215288 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.215292    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw" event={"ID":"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e","Type":"ContainerDied","Data":"0c850c71d58ed258d23ad5be3c0b5e93b25d51ea735dd958e12088a4534a6a90"}
Jul 27 15:41:29.239964 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.239933    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:41:29.273182 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.273121    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:41:29.362797 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.362768    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq97g\" (UniqueName: \"kubernetes.io/projected/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-kube-api-access-hq97g\") pod \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\" (UID: \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\") "
Jul 27 15:41:29.362797 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.362808    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-openshift-service-ca\") pod \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\" (UID: \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\") "
Jul 27 15:41:29.363050 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.362836    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-maas-api-tls\") pod \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\" (UID: \"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e\") "
Jul 27 15:41:29.363234 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.363203    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e" (UID: "8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:41:29.364954 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.364928    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-kube-api-access-hq97g" (OuterVolumeSpecName: "kube-api-access-hq97g") pod "8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e" (UID: "8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e"). InnerVolumeSpecName "kube-api-access-hq97g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:41:29.365572 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.365518    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e" (UID: "8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:41:29.464223 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.464192    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq97g\" (UniqueName: \"kubernetes.io/projected/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-kube-api-access-hq97g\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:41:29.464223 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.464216    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:41:29.464223 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:29.464226    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:41:30.222337 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:30.222306    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"
Jul 27 15:41:30.222337 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:30.222332    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw" event={"ID":"8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e","Type":"ContainerDied","Data":"de9f7b21b19827edfac8e6aa5127b0c8a5c80801bffc6fa48dd42217b5d9e42c"}
Jul 27 15:41:30.222848 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:30.222393    2548 scope.go:117] "RemoveContainer" containerID="0c850c71d58ed258d23ad5be3c0b5e93b25d51ea735dd958e12088a4534a6a90"
Jul 27 15:41:30.246841 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:30.246808    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"]
Jul 27 15:41:30.251214 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:30.251189    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-44b2dff2-556c9cbb7-9pwfw"]
Jul 27 15:41:31.138966 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:31.138933    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e" path="/var/lib/kubelet/pods/8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e/volumes"
Jul 27 15:41:33.410605 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:33.410566    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d"]
Jul 27 15:41:33.415233 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:33.415208    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44b2dff2-lmw5d"]
Jul 27 15:41:33.416239 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:33.416218    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:41:35.138228 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:35.138181    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e5601c5-21de-43cc-b319-32efd8669cd9" path="/var/lib/kubelet/pods/7e5601c5-21de-43cc-b319-32efd8669cd9/volumes"
Jul 27 15:41:36.652046 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:36.652014    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:41:42.136543 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:41:42.136510    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:18.328058 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:18.328024    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:19.258239 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:19.258205    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:25.147941 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:25.147907    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:25.216970 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:25.216935    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:25.314236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:25.314207    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:25.537030 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:25.536978    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:26.656818 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:26.656790    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:26.778488 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:26.778457    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:27.584471 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:27.584443    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:27.683930 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:27.683904    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:27.782875 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:27.782843    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:29.188304 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:29.188267    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:29.348410 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:29.348375    2548 scope.go:117] "RemoveContainer" containerID="1777f0dff6d5deeb20dc70a733f2188bf06c74ec891d3db4c271a1e4b70952be"
Jul 27 15:42:35.321984 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:35.321944    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:35.405280 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:35.405244    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:35.982838 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:35.982804    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:43.079967 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:43.079936    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:43.155131 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:43.155101    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:43.292962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:43.292930    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:43.401594 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:43.401490    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:52.902593 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:52.902482    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:53.836726 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:53.836683    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:42:54.186802 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:42:54.186701    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:15.162205 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:15.162170    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:21.827291 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:21.827259    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:21.914695 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:21.914658    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:22.182298 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:22.182203    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:22.496053 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:22.496022    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:22.580520 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:22.580485    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:25.250281 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.250249    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:25.348589 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.348552    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:25.454256 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.454222    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"]
Jul 27 15:43:25.454813 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.454794    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e" containerName="maas-api"
Jul 27 15:43:25.454948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.454814    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e" containerName="maas-api"
Jul 27 15:43:25.454948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.454848    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e5601c5-21de-43cc-b319-32efd8669cd9" containerName="revoke-keys"
Jul 27 15:43:25.454948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.454857    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e5601c5-21de-43cc-b319-32efd8669cd9" containerName="revoke-keys"
Jul 27 15:43:25.454948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.454948    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fa6f30f-d8e7-4d6e-b0ab-7618a8e2466e" containerName="maas-api"
Jul 27 15:43:25.455083 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.454958    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e5601c5-21de-43cc-b319-32efd8669cd9" containerName="revoke-keys"
Jul 27 15:43:25.458402 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.458383    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:43:25.461081 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.461063    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-0d2dee-serving-cert\""
Jul 27 15:43:25.465405 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.465147    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"]
Jul 27 15:43:25.498948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.498912    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9006afd1-3584-4c18-a156-6645ccf0f5ee-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8\" (UID: \"9006afd1-3584-4c18-a156-6645ccf0f5ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:43:25.499111 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.499045    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2v72\" (UniqueName: \"kubernetes.io/projected/9006afd1-3584-4c18-a156-6645ccf0f5ee-kube-api-access-s2v72\") pod \"maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8\" (UID: \"9006afd1-3584-4c18-a156-6645ccf0f5ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:43:25.499111 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.499081    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9006afd1-3584-4c18-a156-6645ccf0f5ee-maas-api-tls\") pod \"maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8\" (UID: \"9006afd1-3584-4c18-a156-6645ccf0f5ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:43:25.600223 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.600119    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9006afd1-3584-4c18-a156-6645ccf0f5ee-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8\" (UID: \"9006afd1-3584-4c18-a156-6645ccf0f5ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:43:25.600223 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.600211    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2v72\" (UniqueName: \"kubernetes.io/projected/9006afd1-3584-4c18-a156-6645ccf0f5ee-kube-api-access-s2v72\") pod \"maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8\" (UID: \"9006afd1-3584-4c18-a156-6645ccf0f5ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:43:25.600463 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.600232    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9006afd1-3584-4c18-a156-6645ccf0f5ee-maas-api-tls\") pod \"maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8\" (UID: \"9006afd1-3584-4c18-a156-6645ccf0f5ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:43:25.600805 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.600780    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9006afd1-3584-4c18-a156-6645ccf0f5ee-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8\" (UID: \"9006afd1-3584-4c18-a156-6645ccf0f5ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:43:25.602785 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.602766    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9006afd1-3584-4c18-a156-6645ccf0f5ee-maas-api-tls\") pod \"maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8\" (UID: \"9006afd1-3584-4c18-a156-6645ccf0f5ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:43:25.608844 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.608824    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2v72\" (UniqueName: \"kubernetes.io/projected/9006afd1-3584-4c18-a156-6645ccf0f5ee-kube-api-access-s2v72\") pod \"maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8\" (UID: \"9006afd1-3584-4c18-a156-6645ccf0f5ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:43:25.770425 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.770383    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:43:25.914114 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:25.914085    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"]
Jul 27 15:43:25.916177 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:43:25.916152    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9006afd1_3584_4c18_a156_6645ccf0f5ee.slice/crio-402540c3506ac64f93a0180c74818db740a4db885dac8c523e6ee2b677bcb18d WatchSource:0}: Error finding container 402540c3506ac64f93a0180c74818db740a4db885dac8c523e6ee2b677bcb18d: Status 404 returned error can't find the container with id 402540c3506ac64f93a0180c74818db740a4db885dac8c523e6ee2b677bcb18d
Jul 27 15:43:26.115646 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.115574    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"]
Jul 27 15:43:26.120122 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.120099    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"
Jul 27 15:43:26.125157 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.125136    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-0d2dee\""
Jul 27 15:43:26.125335 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.125322    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-0d2dee-dockercfg-9jjjv\""
Jul 27 15:43:26.130168 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.130145    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"]
Jul 27 15:43:26.205594 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.205548    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fd5faac-3b32-4deb-a35b-0489b94fdde1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf\" (UID: \"9fd5faac-3b32-4deb-a35b-0489b94fdde1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"
Jul 27 15:43:26.205813 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.205693    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvdjj\" (UniqueName: \"kubernetes.io/projected/9fd5faac-3b32-4deb-a35b-0489b94fdde1-kube-api-access-nvdjj\") pod \"payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf\" (UID: \"9fd5faac-3b32-4deb-a35b-0489b94fdde1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"
Jul 27 15:43:26.306605 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.306566    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvdjj\" (UniqueName: \"kubernetes.io/projected/9fd5faac-3b32-4deb-a35b-0489b94fdde1-kube-api-access-nvdjj\") pod \"payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf\" (UID: \"9fd5faac-3b32-4deb-a35b-0489b94fdde1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"
Jul 27 15:43:26.307021 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.306636    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fd5faac-3b32-4deb-a35b-0489b94fdde1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf\" (UID: \"9fd5faac-3b32-4deb-a35b-0489b94fdde1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"
Jul 27 15:43:26.307307 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.307288    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fd5faac-3b32-4deb-a35b-0489b94fdde1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf\" (UID: \"9fd5faac-3b32-4deb-a35b-0489b94fdde1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"
Jul 27 15:43:26.315666 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.315642    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvdjj\" (UniqueName: \"kubernetes.io/projected/9fd5faac-3b32-4deb-a35b-0489b94fdde1-kube-api-access-nvdjj\") pod \"payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf\" (UID: \"9fd5faac-3b32-4deb-a35b-0489b94fdde1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"
Jul 27 15:43:26.432220 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.432197    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"
Jul 27 15:43:26.432640 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.432620    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:26.567706 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.567678    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"]
Jul 27 15:43:26.569349 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:43:26.569323    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd5faac_3b32_4deb_a35b_0489b94fdde1.slice/crio-ce295c84db287cec2ccb1180f70a598d3971491f6dc9f630beb00cb7623dbd10 WatchSource:0}: Error finding container ce295c84db287cec2ccb1180f70a598d3971491f6dc9f630beb00cb7623dbd10: Status 404 returned error can't find the container with id ce295c84db287cec2ccb1180f70a598d3971491f6dc9f630beb00cb7623dbd10
Jul 27 15:43:26.789018 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.788974    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf" event={"ID":"9fd5faac-3b32-4deb-a35b-0489b94fdde1","Type":"ContainerStarted","Data":"84a348a2a4b5bd512ba3738a3c1ad50f380e5f35eb63d3e6e2a257d57781ee85"}
Jul 27 15:43:26.789216 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.789047    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf" event={"ID":"9fd5faac-3b32-4deb-a35b-0489b94fdde1","Type":"ContainerStarted","Data":"ce295c84db287cec2ccb1180f70a598d3971491f6dc9f630beb00cb7623dbd10"}
Jul 27 15:43:26.789216 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.789077    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"
Jul 27 15:43:26.790854 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.790830    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8" event={"ID":"9006afd1-3584-4c18-a156-6645ccf0f5ee","Type":"ContainerStarted","Data":"bcb8a78143e4db8d04833e1d57811a2ae11b0e6c8805ab48c4e0d49f334a67c7"}
Jul 27 15:43:26.790989 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.790876    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8" event={"ID":"9006afd1-3584-4c18-a156-6645ccf0f5ee","Type":"ContainerStarted","Data":"402540c3506ac64f93a0180c74818db740a4db885dac8c523e6ee2b677bcb18d"}
Jul 27 15:43:26.791065 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.791026    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:43:26.811673 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.811627    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf" podStartSLOduration=0.811612632 podStartE2EDuration="811.612632ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:43:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:43:26.809876371 +0000 UTC m=+2278.238551555" watchObservedRunningTime="2026-07-27 15:43:26.811612632 +0000 UTC m=+2278.240287815"
Jul 27 15:43:26.827500 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:26.827459    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8" podStartSLOduration=1.389605945 podStartE2EDuration="1.827447273s" totalImagesPullingTime="437.841328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:43:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:43:26.826273417 +0000 UTC m=+2278.254948600" watchObservedRunningTime="2026-07-27 15:43:26.827447273 +0000 UTC m=+2278.256122456"
Jul 27 15:43:27.528877 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:27.528842    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:29.374406 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:29.374378    2548 scope.go:117] "RemoveContainer" containerID="97b2ec9f057e8bd3a3db830ebaa9ea9ac88a1d1d41090c4cea05d6d4e65f87fb"
Jul 27 15:43:29.614793 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:29.614760    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:29.958089 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:29.958059    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:30.289790 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:30.289742    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:30.606894 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:30.606813    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:32.801882 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:32.801854    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:43:33.148498 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.148397    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:33.222464 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.222426    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"]
Jul 27 15:43:33.225470 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.225422    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:33.225864 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.225849    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:43:33.228872 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.228846    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-664c72-serving-cert\""
Jul 27 15:43:33.236102 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.236074    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"]
Jul 27 15:43:33.299347 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.299320    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:33.370984 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.370956    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c94383bf-31ce-492a-bc04-ccb9a37e9a43-maas-api-tls\") pod \"maas-api-e2e-inf-b-664c72-6996b4f84-96mq4\" (UID: \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:43:33.371165 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.371081    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c94383bf-31ce-492a-bc04-ccb9a37e9a43-openshift-service-ca\") pod \"maas-api-e2e-inf-b-664c72-6996b4f84-96mq4\" (UID: \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:43:33.371165 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.371120    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx5t6\" (UniqueName: \"kubernetes.io/projected/c94383bf-31ce-492a-bc04-ccb9a37e9a43-kube-api-access-qx5t6\") pod \"maas-api-e2e-inf-b-664c72-6996b4f84-96mq4\" (UID: \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:43:33.472165 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.472075    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c94383bf-31ce-492a-bc04-ccb9a37e9a43-openshift-service-ca\") pod \"maas-api-e2e-inf-b-664c72-6996b4f84-96mq4\" (UID: \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:43:33.472165 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.472115    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx5t6\" (UniqueName: \"kubernetes.io/projected/c94383bf-31ce-492a-bc04-ccb9a37e9a43-kube-api-access-qx5t6\") pod \"maas-api-e2e-inf-b-664c72-6996b4f84-96mq4\" (UID: \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:43:33.472389 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.472240    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c94383bf-31ce-492a-bc04-ccb9a37e9a43-maas-api-tls\") pod \"maas-api-e2e-inf-b-664c72-6996b4f84-96mq4\" (UID: \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:43:33.472389 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:43:33.472374    2548 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-serving-cert: secret "maas-api-e2e-inf-b-664c72-serving-cert" not found
Jul 27 15:43:33.472492 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:43:33.472446    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c94383bf-31ce-492a-bc04-ccb9a37e9a43-maas-api-tls podName:c94383bf-31ce-492a-bc04-ccb9a37e9a43 nodeName:}" failed. No retries permitted until 2026-07-27 15:43:33.972429365 +0000 UTC m=+2285.401104531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c94383bf-31ce-492a-bc04-ccb9a37e9a43-maas-api-tls") pod "maas-api-e2e-inf-b-664c72-6996b4f84-96mq4" (UID: "c94383bf-31ce-492a-bc04-ccb9a37e9a43") : secret "maas-api-e2e-inf-b-664c72-serving-cert" not found
Jul 27 15:43:33.472727 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.472709    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c94383bf-31ce-492a-bc04-ccb9a37e9a43-openshift-service-ca\") pod \"maas-api-e2e-inf-b-664c72-6996b4f84-96mq4\" (UID: \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:43:33.482393 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.482369    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx5t6\" (UniqueName: \"kubernetes.io/projected/c94383bf-31ce-492a-bc04-ccb9a37e9a43-kube-api-access-qx5t6\") pod \"maas-api-e2e-inf-b-664c72-6996b4f84-96mq4\" (UID: \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:43:33.555004 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.554966    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:33.826630 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.826596    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:33.924396 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.924355    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:33.977956 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.977917    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c94383bf-31ce-492a-bc04-ccb9a37e9a43-maas-api-tls\") pod \"maas-api-e2e-inf-b-664c72-6996b4f84-96mq4\" (UID: \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:43:33.980459 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:33.980436    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c94383bf-31ce-492a-bc04-ccb9a37e9a43-maas-api-tls\") pod \"maas-api-e2e-inf-b-664c72-6996b4f84-96mq4\" (UID: \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:43:34.137822 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:34.137738    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:43:34.277716 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:34.277682    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"]
Jul 27 15:43:34.279244 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:43:34.279211    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc94383bf_31ce_492a_bc04_ccb9a37e9a43.slice/crio-156604e100e5a78ebafa44c7a2ce207265568259931af65bae48e209bd848d97 WatchSource:0}: Error finding container 156604e100e5a78ebafa44c7a2ce207265568259931af65bae48e209bd848d97: Status 404 returned error can't find the container with id 156604e100e5a78ebafa44c7a2ce207265568259931af65bae48e209bd848d97
Jul 27 15:43:34.839869 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:34.839781    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4" event={"ID":"c94383bf-31ce-492a-bc04-ccb9a37e9a43","Type":"ContainerStarted","Data":"41e81f57bcdf6ac86503d1540c5729ec3527e28b866c5a5cfaa68f1b0ce27c22"}
Jul 27 15:43:34.839869 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:34.839845    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:43:34.839869 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:34.839862    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4" event={"ID":"c94383bf-31ce-492a-bc04-ccb9a37e9a43","Type":"ContainerStarted","Data":"156604e100e5a78ebafa44c7a2ce207265568259931af65bae48e209bd848d97"}
Jul 27 15:43:34.857626 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:34.857569    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4" podStartSLOduration=1.5865903540000001 podStartE2EDuration="1.857524294s" totalImagesPullingTime="270.93394ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:43:34.856466977 +0000 UTC m=+2286.285142185" watchObservedRunningTime="2026-07-27 15:43:34.857524294 +0000 UTC m=+2286.286199477"
Jul 27 15:43:37.690305 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:37.690260    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:43:37.798707 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:37.798680    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"
Jul 27 15:43:40.849303 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:40.849275    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:43:45.524073 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:43:45.524041    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:44:02.004598 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:02.004565    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:44:47.047641 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:47.047600    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:44:49.090805 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:49.090768    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:44:58.453222 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.453185    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"]
Jul 27 15:44:58.453716 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.453506    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8" podUID="9006afd1-3584-4c18-a156-6645ccf0f5ee" containerName="maas-api" containerID="cri-o://bcb8a78143e4db8d04833e1d57811a2ae11b0e6c8805ab48c4e0d49f334a67c7" gracePeriod=30
Jul 27 15:44:58.479311 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.479282    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:44:58.505790 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.505759    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"]
Jul 27 15:44:58.506053 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.506004    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf" podUID="9fd5faac-3b32-4deb-a35b-0489b94fdde1" containerName="payload-processing" containerID="cri-o://84a348a2a4b5bd512ba3738a3c1ad50f380e5f35eb63d3e6e2a257d57781ee85" gracePeriod=30
Jul 27 15:44:58.579903 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.579856    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:44:58.650397 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.650346    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:44:58.758587 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.758562    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:44:58.796847 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.796826    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"
Jul 27 15:44:58.837163 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.837137    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvdjj\" (UniqueName: \"kubernetes.io/projected/9fd5faac-3b32-4deb-a35b-0489b94fdde1-kube-api-access-nvdjj\") pod \"9fd5faac-3b32-4deb-a35b-0489b94fdde1\" (UID: \"9fd5faac-3b32-4deb-a35b-0489b94fdde1\") "
Jul 27 15:44:58.837342 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.837183    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2v72\" (UniqueName: \"kubernetes.io/projected/9006afd1-3584-4c18-a156-6645ccf0f5ee-kube-api-access-s2v72\") pod \"9006afd1-3584-4c18-a156-6645ccf0f5ee\" (UID: \"9006afd1-3584-4c18-a156-6645ccf0f5ee\") "
Jul 27 15:44:58.837342 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.837203    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fd5faac-3b32-4deb-a35b-0489b94fdde1-plugins-config-volume\") pod \"9fd5faac-3b32-4deb-a35b-0489b94fdde1\" (UID: \"9fd5faac-3b32-4deb-a35b-0489b94fdde1\") "
Jul 27 15:44:58.837342 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.837225    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9006afd1-3584-4c18-a156-6645ccf0f5ee-openshift-service-ca\") pod \"9006afd1-3584-4c18-a156-6645ccf0f5ee\" (UID: \"9006afd1-3584-4c18-a156-6645ccf0f5ee\") "
Jul 27 15:44:58.837342 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.837311    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9006afd1-3584-4c18-a156-6645ccf0f5ee-maas-api-tls\") pod \"9006afd1-3584-4c18-a156-6645ccf0f5ee\" (UID: \"9006afd1-3584-4c18-a156-6645ccf0f5ee\") "
Jul 27 15:44:58.837618 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.837586    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fd5faac-3b32-4deb-a35b-0489b94fdde1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9fd5faac-3b32-4deb-a35b-0489b94fdde1" (UID: "9fd5faac-3b32-4deb-a35b-0489b94fdde1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:44:58.837682 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.837656    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9006afd1-3584-4c18-a156-6645ccf0f5ee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9006afd1-3584-4c18-a156-6645ccf0f5ee" (UID: "9006afd1-3584-4c18-a156-6645ccf0f5ee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:44:58.839507 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.839467    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9006afd1-3584-4c18-a156-6645ccf0f5ee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9006afd1-3584-4c18-a156-6645ccf0f5ee" (UID: "9006afd1-3584-4c18-a156-6645ccf0f5ee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:44:58.839647 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.839585    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9006afd1-3584-4c18-a156-6645ccf0f5ee-kube-api-access-s2v72" (OuterVolumeSpecName: "kube-api-access-s2v72") pod "9006afd1-3584-4c18-a156-6645ccf0f5ee" (UID: "9006afd1-3584-4c18-a156-6645ccf0f5ee"). InnerVolumeSpecName "kube-api-access-s2v72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:44:58.839709 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.839647    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fd5faac-3b32-4deb-a35b-0489b94fdde1-kube-api-access-nvdjj" (OuterVolumeSpecName: "kube-api-access-nvdjj") pod "9fd5faac-3b32-4deb-a35b-0489b94fdde1" (UID: "9fd5faac-3b32-4deb-a35b-0489b94fdde1"). InnerVolumeSpecName "kube-api-access-nvdjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:44:58.938430 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.938401    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9006afd1-3584-4c18-a156-6645ccf0f5ee-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:44:58.938430 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.938427    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvdjj\" (UniqueName: \"kubernetes.io/projected/9fd5faac-3b32-4deb-a35b-0489b94fdde1-kube-api-access-nvdjj\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:44:58.938430 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.938436    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2v72\" (UniqueName: \"kubernetes.io/projected/9006afd1-3584-4c18-a156-6645ccf0f5ee-kube-api-access-s2v72\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:44:58.938667 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.938444    2548 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fd5faac-3b32-4deb-a35b-0489b94fdde1-plugins-config-volume\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:44:58.938667 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:58.938452    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9006afd1-3584-4c18-a156-6645ccf0f5ee-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:44:59.251759 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.251719    2548 generic.go:356] "Generic (PLEG): container finished" podID="9fd5faac-3b32-4deb-a35b-0489b94fdde1" containerID="84a348a2a4b5bd512ba3738a3c1ad50f380e5f35eb63d3e6e2a257d57781ee85" exitCode=0
Jul 27 15:44:59.251932 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.251767    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"
Jul 27 15:44:59.251932 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.251809    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf" event={"ID":"9fd5faac-3b32-4deb-a35b-0489b94fdde1","Type":"ContainerDied","Data":"84a348a2a4b5bd512ba3738a3c1ad50f380e5f35eb63d3e6e2a257d57781ee85"}
Jul 27 15:44:59.251932 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.251868    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf" event={"ID":"9fd5faac-3b32-4deb-a35b-0489b94fdde1","Type":"ContainerDied","Data":"ce295c84db287cec2ccb1180f70a598d3971491f6dc9f630beb00cb7623dbd10"}
Jul 27 15:44:59.251932 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.251896    2548 scope.go:117] "RemoveContainer" containerID="84a348a2a4b5bd512ba3738a3c1ad50f380e5f35eb63d3e6e2a257d57781ee85"
Jul 27 15:44:59.253229 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.253204    2548 generic.go:356] "Generic (PLEG): container finished" podID="9006afd1-3584-4c18-a156-6645ccf0f5ee" containerID="bcb8a78143e4db8d04833e1d57811a2ae11b0e6c8805ab48c4e0d49f334a67c7" exitCode=0
Jul 27 15:44:59.253351 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.253248    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"
Jul 27 15:44:59.253351 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.253294    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8" event={"ID":"9006afd1-3584-4c18-a156-6645ccf0f5ee","Type":"ContainerDied","Data":"bcb8a78143e4db8d04833e1d57811a2ae11b0e6c8805ab48c4e0d49f334a67c7"}
Jul 27 15:44:59.253351 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.253339    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8" event={"ID":"9006afd1-3584-4c18-a156-6645ccf0f5ee","Type":"ContainerDied","Data":"402540c3506ac64f93a0180c74818db740a4db885dac8c523e6ee2b677bcb18d"}
Jul 27 15:44:59.260823 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.260807    2548 scope.go:117] "RemoveContainer" containerID="84a348a2a4b5bd512ba3738a3c1ad50f380e5f35eb63d3e6e2a257d57781ee85"
Jul 27 15:44:59.261063 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:44:59.261047    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84a348a2a4b5bd512ba3738a3c1ad50f380e5f35eb63d3e6e2a257d57781ee85\": container with ID starting with 84a348a2a4b5bd512ba3738a3c1ad50f380e5f35eb63d3e6e2a257d57781ee85 not found: ID does not exist" containerID="84a348a2a4b5bd512ba3738a3c1ad50f380e5f35eb63d3e6e2a257d57781ee85"
Jul 27 15:44:59.261126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.261068    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84a348a2a4b5bd512ba3738a3c1ad50f380e5f35eb63d3e6e2a257d57781ee85"} err="failed to get container status \"84a348a2a4b5bd512ba3738a3c1ad50f380e5f35eb63d3e6e2a257d57781ee85\": rpc error: code = NotFound desc = could not find container \"84a348a2a4b5bd512ba3738a3c1ad50f380e5f35eb63d3e6e2a257d57781ee85\": container with ID starting with 84a348a2a4b5bd512ba3738a3c1ad50f380e5f35eb63d3e6e2a257d57781ee85 not found: ID does not exist"
Jul 27 15:44:59.261126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.261083    2548 scope.go:117] "RemoveContainer" containerID="bcb8a78143e4db8d04833e1d57811a2ae11b0e6c8805ab48c4e0d49f334a67c7"
Jul 27 15:44:59.268765 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.268750    2548 scope.go:117] "RemoveContainer" containerID="bcb8a78143e4db8d04833e1d57811a2ae11b0e6c8805ab48c4e0d49f334a67c7"
Jul 27 15:44:59.269004 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:44:59.268985    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcb8a78143e4db8d04833e1d57811a2ae11b0e6c8805ab48c4e0d49f334a67c7\": container with ID starting with bcb8a78143e4db8d04833e1d57811a2ae11b0e6c8805ab48c4e0d49f334a67c7 not found: ID does not exist" containerID="bcb8a78143e4db8d04833e1d57811a2ae11b0e6c8805ab48c4e0d49f334a67c7"
Jul 27 15:44:59.269083 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.269009    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcb8a78143e4db8d04833e1d57811a2ae11b0e6c8805ab48c4e0d49f334a67c7"} err="failed to get container status \"bcb8a78143e4db8d04833e1d57811a2ae11b0e6c8805ab48c4e0d49f334a67c7\": rpc error: code = NotFound desc = could not find container \"bcb8a78143e4db8d04833e1d57811a2ae11b0e6c8805ab48c4e0d49f334a67c7\": container with ID starting with bcb8a78143e4db8d04833e1d57811a2ae11b0e6c8805ab48c4e0d49f334a67c7 not found: ID does not exist"
Jul 27 15:44:59.269760 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.269740    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"]
Jul 27 15:44:59.273331 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.273311    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0d2dee-7f96b798f9-t9bz8"]
Jul 27 15:44:59.284684 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.284659    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"]
Jul 27 15:44:59.287151 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:44:59.287131    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0d2dee-5f79b46b46-7b4xf"]
Jul 27 15:45:00.133836 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.133806    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752785-6k5gq"]
Jul 27 15:45:00.134259 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.134240    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9006afd1-3584-4c18-a156-6645ccf0f5ee" containerName="maas-api"
Jul 27 15:45:00.134259 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.134257    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="9006afd1-3584-4c18-a156-6645ccf0f5ee" containerName="maas-api"
Jul 27 15:45:00.134259 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.134267    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fd5faac-3b32-4deb-a35b-0489b94fdde1" containerName="payload-processing"
Jul 27 15:45:00.134259 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.134274    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fd5faac-3b32-4deb-a35b-0489b94fdde1" containerName="payload-processing"
Jul 27 15:45:00.134642 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.134381    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="9006afd1-3584-4c18-a156-6645ccf0f5ee" containerName="maas-api"
Jul 27 15:45:00.134642 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.134397    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fd5faac-3b32-4deb-a35b-0489b94fdde1" containerName="payload-processing"
Jul 27 15:45:00.139034 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.139014    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-664c72-29752785-cwznd"]
Jul 27 15:45:00.139144 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.139126    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752785-6k5gq"
Jul 27 15:45:00.142027 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.142003    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bgfwg\""
Jul 27 15:45:00.142458 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.142411    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-664c72-29752785-cwznd"
Jul 27 15:45:00.144708 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.144498    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752785-6k5gq"]
Jul 27 15:45:00.147719 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.147698    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-664c72-29752785-cwznd"]
Jul 27 15:45:00.154082 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.154063    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752785-6k5gq"
Jul 27 15:45:00.160795 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.160779    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-664c72-29752785-cwznd"
Jul 27 15:45:00.303011 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.302929    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752785-6k5gq"]
Jul 27 15:45:00.305819 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:45:00.305783    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9442ae1_7269_45e9_8039_2a1e1e75216f.slice/crio-0dd48ccbe63f3a73f49bfa37057d5c1c4d1a53e4470ddc0a236af5e7482a3e0d WatchSource:0}: Error finding container 0dd48ccbe63f3a73f49bfa37057d5c1c4d1a53e4470ddc0a236af5e7482a3e0d: Status 404 returned error can't find the container with id 0dd48ccbe63f3a73f49bfa37057d5c1c4d1a53e4470ddc0a236af5e7482a3e0d
Jul 27 15:45:00.336637 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:00.336611    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-664c72-29752785-cwznd"]
Jul 27 15:45:00.337917 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:45:00.337894    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8a1263c_d73e_4b51_9888_d46aede14cea.slice/crio-4dd58196df8d9998ae938fdb51c773fa65648f1199656c02452aba2c01e9fcbc WatchSource:0}: Error finding container 4dd58196df8d9998ae938fdb51c773fa65648f1199656c02452aba2c01e9fcbc: Status 404 returned error can't find the container with id 4dd58196df8d9998ae938fdb51c773fa65648f1199656c02452aba2c01e9fcbc
Jul 27 15:45:01.139691 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:01.139654    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9006afd1-3584-4c18-a156-6645ccf0f5ee" path="/var/lib/kubelet/pods/9006afd1-3584-4c18-a156-6645ccf0f5ee/volumes"
Jul 27 15:45:01.140057 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:01.140026    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fd5faac-3b32-4deb-a35b-0489b94fdde1" path="/var/lib/kubelet/pods/9fd5faac-3b32-4deb-a35b-0489b94fdde1/volumes"
Jul 27 15:45:01.269105 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:01.269071    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-664c72-29752785-cwznd" event={"ID":"a8a1263c-d73e-4b51-9888-d46aede14cea","Type":"ContainerStarted","Data":"c636c55755f6aebac2a0d34dab52921e8605cbfb7c1c6a4d21be5a401ad62d42"}
Jul 27 15:45:01.269280 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:01.269122    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-664c72-29752785-cwznd" event={"ID":"a8a1263c-d73e-4b51-9888-d46aede14cea","Type":"ContainerStarted","Data":"4dd58196df8d9998ae938fdb51c773fa65648f1199656c02452aba2c01e9fcbc"}
Jul 27 15:45:01.270600 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:01.270574    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752785-6k5gq" event={"ID":"a9442ae1-7269-45e9-8039-2a1e1e75216f","Type":"ContainerStarted","Data":"51f1e449bc3c70b78eab97de63565dd1d6b2800cfe8318d79a83c2e07a51a8b5"}
Jul 27 15:45:01.270745 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:01.270616    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752785-6k5gq" event={"ID":"a9442ae1-7269-45e9-8039-2a1e1e75216f","Type":"ContainerStarted","Data":"0dd48ccbe63f3a73f49bfa37057d5c1c4d1a53e4470ddc0a236af5e7482a3e0d"}
Jul 27 15:45:01.286474 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:01.286428    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-664c72-29752785-cwznd" podStartSLOduration=1.286414554 podStartE2EDuration="1.286414554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:45:01.283651465 +0000 UTC m=+2372.712326650" watchObservedRunningTime="2026-07-27 15:45:01.286414554 +0000 UTC m=+2372.715089738"
Jul 27 15:45:01.301281 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:01.301238    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752785-6k5gq" podStartSLOduration=1.301223041 podStartE2EDuration="1.301223041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:45:01.299259916 +0000 UTC m=+2372.727935099" watchObservedRunningTime="2026-07-27 15:45:01.301223041 +0000 UTC m=+2372.729898223"
Jul 27 15:45:02.277512 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:02.277479    2548 generic.go:356] "Generic (PLEG): container finished" podID="a9442ae1-7269-45e9-8039-2a1e1e75216f" containerID="51f1e449bc3c70b78eab97de63565dd1d6b2800cfe8318d79a83c2e07a51a8b5" exitCode=0
Jul 27 15:45:02.277950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:02.277577    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752785-6k5gq" event={"ID":"a9442ae1-7269-45e9-8039-2a1e1e75216f","Type":"ContainerDied","Data":"51f1e449bc3c70b78eab97de63565dd1d6b2800cfe8318d79a83c2e07a51a8b5"}
Jul 27 15:45:02.279131 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:02.279109    2548 generic.go:356] "Generic (PLEG): container finished" podID="a8a1263c-d73e-4b51-9888-d46aede14cea" containerID="c636c55755f6aebac2a0d34dab52921e8605cbfb7c1c6a4d21be5a401ad62d42" exitCode=0
Jul 27 15:45:02.279240 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:02.279160    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-664c72-29752785-cwznd" event={"ID":"a8a1263c-d73e-4b51-9888-d46aede14cea","Type":"ContainerDied","Data":"c636c55755f6aebac2a0d34dab52921e8605cbfb7c1c6a4d21be5a401ad62d42"}
Jul 27 15:45:03.137795 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:03.137757    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:03.411045 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:03.411023    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-664c72-29752785-cwznd"
Jul 27 15:45:03.446899 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:03.446869    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752785-6k5gq"
Jul 27 15:45:03.537689 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:03.537658    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:04.291283 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:04.291252    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752785-6k5gq"
Jul 27 15:45:04.291475 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:04.291288    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752785-6k5gq" event={"ID":"a9442ae1-7269-45e9-8039-2a1e1e75216f","Type":"ContainerDied","Data":"0dd48ccbe63f3a73f49bfa37057d5c1c4d1a53e4470ddc0a236af5e7482a3e0d"}
Jul 27 15:45:04.291475 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:04.291335    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0dd48ccbe63f3a73f49bfa37057d5c1c4d1a53e4470ddc0a236af5e7482a3e0d"
Jul 27 15:45:04.292590 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:04.292565    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-664c72-29752785-cwznd" event={"ID":"a8a1263c-d73e-4b51-9888-d46aede14cea","Type":"ContainerDied","Data":"4dd58196df8d9998ae938fdb51c773fa65648f1199656c02452aba2c01e9fcbc"}
Jul 27 15:45:04.292590 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:04.292586    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-664c72-29752785-cwznd"
Jul 27 15:45:04.292775 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:04.292609    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4dd58196df8d9998ae938fdb51c773fa65648f1199656c02452aba2c01e9fcbc"
Jul 27 15:45:09.878379 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:09.878342    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:09.948723 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:09.948693    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:10.145248 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.145159    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:10.399232 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.399145    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:10.510617 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.510586    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c"]
Jul 27 15:45:10.511045 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.511027    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8a1263c-d73e-4b51-9888-d46aede14cea" containerName="cleanup"
Jul 27 15:45:10.511126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.511049    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8a1263c-d73e-4b51-9888-d46aede14cea" containerName="cleanup"
Jul 27 15:45:10.511126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.511075    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9442ae1-7269-45e9-8039-2a1e1e75216f" containerName="cleanup"
Jul 27 15:45:10.511126 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.511084    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9442ae1-7269-45e9-8039-2a1e1e75216f" containerName="cleanup"
Jul 27 15:45:10.511284 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.511183    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9442ae1-7269-45e9-8039-2a1e1e75216f" containerName="cleanup"
Jul 27 15:45:10.511284 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.511198    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8a1263c-d73e-4b51-9888-d46aede14cea" containerName="cleanup"
Jul 27 15:45:10.514179 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.514158    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c"
Jul 27 15:45:10.516703 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.516684    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bgfwg\""
Jul 27 15:45:10.523447 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.523422    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c"]
Jul 27 15:45:10.651967 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.651888    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81cd5ac8-7648-4078-bd61-715dd855483d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-664c72-twz4c\" (UID: \"81cd5ac8-7648-4078-bd61-715dd855483d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c"
Jul 27 15:45:10.752795 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.752759    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81cd5ac8-7648-4078-bd61-715dd855483d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-664c72-twz4c\" (UID: \"81cd5ac8-7648-4078-bd61-715dd855483d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c"
Jul 27 15:45:10.753419 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.753395    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81cd5ac8-7648-4078-bd61-715dd855483d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-664c72-twz4c\" (UID: \"81cd5ac8-7648-4078-bd61-715dd855483d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c"
Jul 27 15:45:10.825371 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.825341    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c"
Jul 27 15:45:10.961869 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:10.961839    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c"]
Jul 27 15:45:10.965490 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:45:10.965467    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81cd5ac8_7648_4078_bd61_715dd855483d.slice/crio-9713c6066ab05ac0ec1632eab7b546760f1e30bc5e54d3a9e4f7b9ebcdc7631a WatchSource:0}: Error finding container 9713c6066ab05ac0ec1632eab7b546760f1e30bc5e54d3a9e4f7b9ebcdc7631a: Status 404 returned error can't find the container with id 9713c6066ab05ac0ec1632eab7b546760f1e30bc5e54d3a9e4f7b9ebcdc7631a
Jul 27 15:45:11.330461 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:11.330414    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c" event={"ID":"81cd5ac8-7648-4078-bd61-715dd855483d","Type":"ContainerStarted","Data":"be59e99e82d6eea0c82db979c613a8ed91c2b36c37e6a5e30dbaab22b8054edb"}
Jul 27 15:45:11.330655 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:11.330477    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c" event={"ID":"81cd5ac8-7648-4078-bd61-715dd855483d","Type":"ContainerStarted","Data":"9713c6066ab05ac0ec1632eab7b546760f1e30bc5e54d3a9e4f7b9ebcdc7631a"}
Jul 27 15:45:11.354459 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:11.354399    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c" podStartSLOduration=1.35438414 podStartE2EDuration="1.35438414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:45:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:45:11.351613003 +0000 UTC m=+2382.780288187" watchObservedRunningTime="2026-07-27 15:45:11.35438414 +0000 UTC m=+2382.783059322"
Jul 27 15:45:12.337479 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:12.337442    2548 generic.go:356] "Generic (PLEG): container finished" podID="81cd5ac8-7648-4078-bd61-715dd855483d" containerID="be59e99e82d6eea0c82db979c613a8ed91c2b36c37e6a5e30dbaab22b8054edb" exitCode=0
Jul 27 15:45:12.337880 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:12.337484    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c" event={"ID":"81cd5ac8-7648-4078-bd61-715dd855483d","Type":"ContainerDied","Data":"be59e99e82d6eea0c82db979c613a8ed91c2b36c37e6a5e30dbaab22b8054edb"}
Jul 27 15:45:13.476371 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:13.476347    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c"
Jul 27 15:45:13.579663 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:13.579623    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81cd5ac8-7648-4078-bd61-715dd855483d-maas-api-service-ca\") pod \"81cd5ac8-7648-4078-bd61-715dd855483d\" (UID: \"81cd5ac8-7648-4078-bd61-715dd855483d\") "
Jul 27 15:45:13.579982 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:13.579962    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81cd5ac8-7648-4078-bd61-715dd855483d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "81cd5ac8-7648-4078-bd61-715dd855483d" (UID: "81cd5ac8-7648-4078-bd61-715dd855483d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:45:13.681170 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:13.681098    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81cd5ac8-7648-4078-bd61-715dd855483d-maas-api-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:45:14.348302 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:14.348274    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c"
Jul 27 15:45:14.348302 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:14.348300    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c" event={"ID":"81cd5ac8-7648-4078-bd61-715dd855483d","Type":"ContainerDied","Data":"9713c6066ab05ac0ec1632eab7b546760f1e30bc5e54d3a9e4f7b9ebcdc7631a"}
Jul 27 15:45:14.348556 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:14.348346    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9713c6066ab05ac0ec1632eab7b546760f1e30bc5e54d3a9e4f7b9ebcdc7631a"
Jul 27 15:45:20.955886 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:20.955845    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"]
Jul 27 15:45:20.956764 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:20.956712    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4" podUID="c94383bf-31ce-492a-bc04-ccb9a37e9a43" containerName="maas-api" containerID="cri-o://41e81f57bcdf6ac86503d1540c5729ec3527e28b866c5a5cfaa68f1b0ce27c22" gracePeriod=30
Jul 27 15:45:20.983751 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:20.983717    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-664c72-29752785-cwznd"]
Jul 27 15:45:20.987966 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:20.987945    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-664c72-29752785-cwznd"]
Jul 27 15:45:21.022885 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.022856    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:21.088910 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.088872    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:21.140962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.140933    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8a1263c-d73e-4b51-9888-d46aede14cea" path="/var/lib/kubelet/pods/a8a1263c-d73e-4b51-9888-d46aede14cea/volumes"
Jul 27 15:45:21.208971 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.208920    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:45:21.349601 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.349566    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx5t6\" (UniqueName: \"kubernetes.io/projected/c94383bf-31ce-492a-bc04-ccb9a37e9a43-kube-api-access-qx5t6\") pod \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\" (UID: \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\") "
Jul 27 15:45:21.349752 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.349610    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c94383bf-31ce-492a-bc04-ccb9a37e9a43-maas-api-tls\") pod \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\" (UID: \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\") "
Jul 27 15:45:21.349752 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.349736    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c94383bf-31ce-492a-bc04-ccb9a37e9a43-openshift-service-ca\") pod \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\" (UID: \"c94383bf-31ce-492a-bc04-ccb9a37e9a43\") "
Jul 27 15:45:21.350118 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.350079    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c94383bf-31ce-492a-bc04-ccb9a37e9a43-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c94383bf-31ce-492a-bc04-ccb9a37e9a43" (UID: "c94383bf-31ce-492a-bc04-ccb9a37e9a43"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:45:21.351802 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.351781    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c94383bf-31ce-492a-bc04-ccb9a37e9a43-kube-api-access-qx5t6" (OuterVolumeSpecName: "kube-api-access-qx5t6") pod "c94383bf-31ce-492a-bc04-ccb9a37e9a43" (UID: "c94383bf-31ce-492a-bc04-ccb9a37e9a43"). InnerVolumeSpecName "kube-api-access-qx5t6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:45:21.351909 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.351824    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c94383bf-31ce-492a-bc04-ccb9a37e9a43-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c94383bf-31ce-492a-bc04-ccb9a37e9a43" (UID: "c94383bf-31ce-492a-bc04-ccb9a37e9a43"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:45:21.382878 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.382848    2548 generic.go:356] "Generic (PLEG): container finished" podID="c94383bf-31ce-492a-bc04-ccb9a37e9a43" containerID="41e81f57bcdf6ac86503d1540c5729ec3527e28b866c5a5cfaa68f1b0ce27c22" exitCode=0
Jul 27 15:45:21.383008 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.382898    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"
Jul 27 15:45:21.383008 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.382903    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4" event={"ID":"c94383bf-31ce-492a-bc04-ccb9a37e9a43","Type":"ContainerDied","Data":"41e81f57bcdf6ac86503d1540c5729ec3527e28b866c5a5cfaa68f1b0ce27c22"}
Jul 27 15:45:21.383008 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.382944    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4" event={"ID":"c94383bf-31ce-492a-bc04-ccb9a37e9a43","Type":"ContainerDied","Data":"156604e100e5a78ebafa44c7a2ce207265568259931af65bae48e209bd848d97"}
Jul 27 15:45:21.383008 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.382966    2548 scope.go:117] "RemoveContainer" containerID="41e81f57bcdf6ac86503d1540c5729ec3527e28b866c5a5cfaa68f1b0ce27c22"
Jul 27 15:45:21.392439 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.392424    2548 scope.go:117] "RemoveContainer" containerID="41e81f57bcdf6ac86503d1540c5729ec3527e28b866c5a5cfaa68f1b0ce27c22"
Jul 27 15:45:21.392732 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:45:21.392709    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41e81f57bcdf6ac86503d1540c5729ec3527e28b866c5a5cfaa68f1b0ce27c22\": container with ID starting with 41e81f57bcdf6ac86503d1540c5729ec3527e28b866c5a5cfaa68f1b0ce27c22 not found: ID does not exist" containerID="41e81f57bcdf6ac86503d1540c5729ec3527e28b866c5a5cfaa68f1b0ce27c22"
Jul 27 15:45:21.392801 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.392740    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41e81f57bcdf6ac86503d1540c5729ec3527e28b866c5a5cfaa68f1b0ce27c22"} err="failed to get container status \"41e81f57bcdf6ac86503d1540c5729ec3527e28b866c5a5cfaa68f1b0ce27c22\": rpc error: code = NotFound desc = could not find container \"41e81f57bcdf6ac86503d1540c5729ec3527e28b866c5a5cfaa68f1b0ce27c22\": container with ID starting with 41e81f57bcdf6ac86503d1540c5729ec3527e28b866c5a5cfaa68f1b0ce27c22 not found: ID does not exist"
Jul 27 15:45:21.406265 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.406240    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"]
Jul 27 15:45:21.412605 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.412582    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-664c72-6996b4f84-96mq4"]
Jul 27 15:45:21.451412 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.451393    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx5t6\" (UniqueName: \"kubernetes.io/projected/c94383bf-31ce-492a-bc04-ccb9a37e9a43-kube-api-access-qx5t6\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:45:21.451412 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.451412    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c94383bf-31ce-492a-bc04-ccb9a37e9a43-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:45:21.451526 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:21.451420    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c94383bf-31ce-492a-bc04-ccb9a37e9a43-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:45:23.144659 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:23.144623    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c94383bf-31ce-492a-bc04-ccb9a37e9a43" path="/var/lib/kubelet/pods/c94383bf-31ce-492a-bc04-ccb9a37e9a43/volumes"
Jul 27 15:45:25.583527 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:25.583493    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c"]
Jul 27 15:45:25.585786 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:25.585758    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-664c72-twz4c"]
Jul 27 15:45:25.621208 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:25.621180    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:25.967946 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:25.967872    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:27.139139 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:27.139107    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81cd5ac8-7648-4078-bd61-715dd855483d" path="/var/lib/kubelet/pods/81cd5ac8-7648-4078-bd61-715dd855483d/volumes"
Jul 27 15:45:31.783519 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:31.783480    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:31.885067 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:31.885033    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:32.062586 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:32.062484    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:32.296425 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:32.296392    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:32.657790 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:32.657757    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:32.955425 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:32.955343    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:34.553618 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:34.553585    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:35.779763 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.779733    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:35.877644 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.877609    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"]
Jul 27 15:45:35.878178 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.878155    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81cd5ac8-7648-4078-bd61-715dd855483d" containerName="revoke-keys"
Jul 27 15:45:35.878178 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.878175    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="81cd5ac8-7648-4078-bd61-715dd855483d" containerName="revoke-keys"
Jul 27 15:45:35.878330 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.878203    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c94383bf-31ce-492a-bc04-ccb9a37e9a43" containerName="maas-api"
Jul 27 15:45:35.878330 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.878211    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="c94383bf-31ce-492a-bc04-ccb9a37e9a43" containerName="maas-api"
Jul 27 15:45:35.878330 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.878287    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="c94383bf-31ce-492a-bc04-ccb9a37e9a43" containerName="maas-api"
Jul 27 15:45:35.878330 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.878299    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="81cd5ac8-7648-4078-bd61-715dd855483d" containerName="revoke-keys"
Jul 27 15:45:35.879115 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.879087    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:35.881586 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.881567    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:45:35.884409 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.884390    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-fcb199-serving-cert\""
Jul 27 15:45:35.904726 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.904703    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"]
Jul 27 15:45:35.963633 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.963609    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:35.978880 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.978850    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-maas-api-tls\") pod \"maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92\" (UID: \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:45:35.979024 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.978883    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-openshift-service-ca\") pod \"maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92\" (UID: \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:45:35.979024 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:35.978944    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw4pq\" (UniqueName: \"kubernetes.io/projected/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-kube-api-access-kw4pq\") pod \"maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92\" (UID: \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:45:36.080363 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.080285    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-maas-api-tls\") pod \"maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92\" (UID: \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:45:36.080363 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.080322    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-openshift-service-ca\") pod \"maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92\" (UID: \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:45:36.080609 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.080449    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw4pq\" (UniqueName: \"kubernetes.io/projected/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-kube-api-access-kw4pq\") pod \"maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92\" (UID: \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:45:36.080970 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.080950    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-openshift-service-ca\") pod \"maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92\" (UID: \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:45:36.083086 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.083066    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-maas-api-tls\") pod \"maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92\" (UID: \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:45:36.099588 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.099564    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw4pq\" (UniqueName: \"kubernetes.io/projected/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-kube-api-access-kw4pq\") pod \"maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92\" (UID: \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:45:36.135297 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.135273    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:36.192511 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.192485    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:45:36.329365 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.329338    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"]
Jul 27 15:45:36.331461 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:45:36.331400    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f6500e9_d9b4_4cc4_a831_83e85e4c8ffa.slice/crio-73e5947f4aee2b923104e9a49caa6913f77562a0f1184d3848bc94955bc9ac98 WatchSource:0}: Error finding container 73e5947f4aee2b923104e9a49caa6913f77562a0f1184d3848bc94955bc9ac98: Status 404 returned error can't find the container with id 73e5947f4aee2b923104e9a49caa6913f77562a0f1184d3848bc94955bc9ac98
Jul 27 15:45:36.332705 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.332686    2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 15:45:36.461257 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.461217    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92" event={"ID":"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa","Type":"ContainerStarted","Data":"73e5947f4aee2b923104e9a49caa6913f77562a0f1184d3848bc94955bc9ac98"}
Jul 27 15:45:36.484462 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.484430    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:36.570183 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.570148    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:36.623478 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.623401    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"]
Jul 27 15:45:36.627979 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.627965    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"
Jul 27 15:45:36.630651 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.630627    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-fcb199\""
Jul 27 15:45:36.630754 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.630628    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-fcb199-dockercfg-c7ln8\""
Jul 27 15:45:36.640849 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.640827    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"]
Jul 27 15:45:36.685332 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.685307    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df2a71cc-90a5-45c2-b66b-c674b38baf92-plugins-config-volume\") pod \"payload-processing-e2e-api-a-fcb199-695946fb8-zqfch\" (UID: \"df2a71cc-90a5-45c2-b66b-c674b38baf92\") " pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"
Jul 27 15:45:36.685440 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.685347    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8b82\" (UniqueName: \"kubernetes.io/projected/df2a71cc-90a5-45c2-b66b-c674b38baf92-kube-api-access-r8b82\") pod \"payload-processing-e2e-api-a-fcb199-695946fb8-zqfch\" (UID: \"df2a71cc-90a5-45c2-b66b-c674b38baf92\") " pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"
Jul 27 15:45:36.785990 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.785961    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8b82\" (UniqueName: \"kubernetes.io/projected/df2a71cc-90a5-45c2-b66b-c674b38baf92-kube-api-access-r8b82\") pod \"payload-processing-e2e-api-a-fcb199-695946fb8-zqfch\" (UID: \"df2a71cc-90a5-45c2-b66b-c674b38baf92\") " pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"
Jul 27 15:45:36.786396 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.786048    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df2a71cc-90a5-45c2-b66b-c674b38baf92-plugins-config-volume\") pod \"payload-processing-e2e-api-a-fcb199-695946fb8-zqfch\" (UID: \"df2a71cc-90a5-45c2-b66b-c674b38baf92\") " pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"
Jul 27 15:45:36.786736 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.786719    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df2a71cc-90a5-45c2-b66b-c674b38baf92-plugins-config-volume\") pod \"payload-processing-e2e-api-a-fcb199-695946fb8-zqfch\" (UID: \"df2a71cc-90a5-45c2-b66b-c674b38baf92\") " pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"
Jul 27 15:45:36.795659 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.795637    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8b82\" (UniqueName: \"kubernetes.io/projected/df2a71cc-90a5-45c2-b66b-c674b38baf92-kube-api-access-r8b82\") pod \"payload-processing-e2e-api-a-fcb199-695946fb8-zqfch\" (UID: \"df2a71cc-90a5-45c2-b66b-c674b38baf92\") " pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"
Jul 27 15:45:36.938382 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.938361    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"
Jul 27 15:45:36.964743 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:36.964721    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:37.076492 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:37.076471    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"]
Jul 27 15:45:37.078583 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:45:37.078555    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf2a71cc_90a5_45c2_b66b_c674b38baf92.slice/crio-7c3df8f3d5a0fe474c74b63bcbff8366c52a8000c35f145119be32a308443c52 WatchSource:0}: Error finding container 7c3df8f3d5a0fe474c74b63bcbff8366c52a8000c35f145119be32a308443c52: Status 404 returned error can't find the container with id 7c3df8f3d5a0fe474c74b63bcbff8366c52a8000c35f145119be32a308443c52
Jul 27 15:45:37.468022 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:37.467937    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92" event={"ID":"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa","Type":"ContainerStarted","Data":"de8540eb5a9fa67c84bda0c1e9367ad7d7977ec4f714c85ae99f5b48cae0a94d"}
Jul 27 15:45:37.468022 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:37.468003    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:45:37.469486 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:37.469461    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch" event={"ID":"df2a71cc-90a5-45c2-b66b-c674b38baf92","Type":"ContainerStarted","Data":"711346d7af6b1970574f79a278b222239fcd1b9faed8996999fee4bf5de3206b"}
Jul 27 15:45:37.469610 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:37.469504    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch" event={"ID":"df2a71cc-90a5-45c2-b66b-c674b38baf92","Type":"ContainerStarted","Data":"7c3df8f3d5a0fe474c74b63bcbff8366c52a8000c35f145119be32a308443c52"}
Jul 27 15:45:37.469610 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:37.469558    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"
Jul 27 15:45:37.488221 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:37.488174    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92" podStartSLOduration=1.927150441 podStartE2EDuration="2.488160893s" totalImagesPullingTime="561.010452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:45:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:45:37.487392461 +0000 UTC m=+2408.916067660" watchObservedRunningTime="2026-07-27 15:45:37.488160893 +0000 UTC m=+2408.916836074"
Jul 27 15:45:37.506927 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:37.506885    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch" podStartSLOduration=1.50687075 podStartE2EDuration="1.50687075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:45:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:45:37.505912579 +0000 UTC m=+2408.934587763" watchObservedRunningTime="2026-07-27 15:45:37.50687075 +0000 UTC m=+2408.935545932"
Jul 27 15:45:40.316884 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:40.316846    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:40.764955 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:40.764924    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:41.154417 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:41.154323    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:41.447953 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:41.447868    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:43.481327 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:43.481295    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:45:43.801854 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:43.801760    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:43.898719 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:43.898654    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:44.086960 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.086872    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"]
Jul 27 15:45:44.091199 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.091178    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:45:44.094231 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.094202    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-fbf2d4-serving-cert\""
Jul 27 15:45:44.104279 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.104252    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"]
Jul 27 15:45:44.134250 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.134218    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:44.161622 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.161597    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dlvn\" (UniqueName: \"kubernetes.io/projected/be519e10-8139-4a09-8f62-d7fcf70a6287-kube-api-access-7dlvn\") pod \"maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q\" (UID: \"be519e10-8139-4a09-8f62-d7fcf70a6287\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:45:44.161738 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.161641    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be519e10-8139-4a09-8f62-d7fcf70a6287-maas-api-tls\") pod \"maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q\" (UID: \"be519e10-8139-4a09-8f62-d7fcf70a6287\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:45:44.161738 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.161678    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be519e10-8139-4a09-8f62-d7fcf70a6287-openshift-service-ca\") pod \"maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q\" (UID: \"be519e10-8139-4a09-8f62-d7fcf70a6287\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:45:44.262686 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.262655    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dlvn\" (UniqueName: \"kubernetes.io/projected/be519e10-8139-4a09-8f62-d7fcf70a6287-kube-api-access-7dlvn\") pod \"maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q\" (UID: \"be519e10-8139-4a09-8f62-d7fcf70a6287\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:45:44.262830 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.262699    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be519e10-8139-4a09-8f62-d7fcf70a6287-maas-api-tls\") pod \"maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q\" (UID: \"be519e10-8139-4a09-8f62-d7fcf70a6287\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:45:44.262830 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.262738    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be519e10-8139-4a09-8f62-d7fcf70a6287-openshift-service-ca\") pod \"maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q\" (UID: \"be519e10-8139-4a09-8f62-d7fcf70a6287\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:45:44.263422 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.263398    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be519e10-8139-4a09-8f62-d7fcf70a6287-openshift-service-ca\") pod \"maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q\" (UID: \"be519e10-8139-4a09-8f62-d7fcf70a6287\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:45:44.265225 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.265204    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be519e10-8139-4a09-8f62-d7fcf70a6287-maas-api-tls\") pod \"maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q\" (UID: \"be519e10-8139-4a09-8f62-d7fcf70a6287\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:45:44.271740 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.271713    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dlvn\" (UniqueName: \"kubernetes.io/projected/be519e10-8139-4a09-8f62-d7fcf70a6287-kube-api-access-7dlvn\") pod \"maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q\" (UID: \"be519e10-8139-4a09-8f62-d7fcf70a6287\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:45:44.312686 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.312650    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:44.402558 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.402459    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:45:44.570163 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.570140    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"]
Jul 27 15:45:44.574081 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:45:44.574052    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe519e10_8139_4a09_8f62_d7fcf70a6287.slice/crio-6166268b1a2500a9ffb14b09d637f2ed1fdbab58dffc1d742df66554b73c4ea7 WatchSource:0}: Error finding container 6166268b1a2500a9ffb14b09d637f2ed1fdbab58dffc1d742df66554b73c4ea7: Status 404 returned error can't find the container with id 6166268b1a2500a9ffb14b09d637f2ed1fdbab58dffc1d742df66554b73c4ea7
Jul 27 15:45:44.696119 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.696046    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:44.777148 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:44.777117    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:45.146282 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:45.146241    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:45.512620 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:45.512589    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q" event={"ID":"be519e10-8139-4a09-8f62-d7fcf70a6287","Type":"ContainerStarted","Data":"afeb8accb04e5762670db7cc23625895ca8ed2f52e9fa922b24d9247ea82a2c6"}
Jul 27 15:45:45.512813 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:45.512647    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q" event={"ID":"be519e10-8139-4a09-8f62-d7fcf70a6287","Type":"ContainerStarted","Data":"6166268b1a2500a9ffb14b09d637f2ed1fdbab58dffc1d742df66554b73c4ea7"}
Jul 27 15:45:45.512813 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:45.512795    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:45:45.535307 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:45.535261    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q" podStartSLOduration=0.81266282 podStartE2EDuration="1.535245976s" totalImagesPullingTime="722.583156ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:45:45.53348538 +0000 UTC m=+2416.962160573" watchObservedRunningTime="2026-07-27 15:45:45.535245976 +0000 UTC m=+2416.963921213"
Jul 27 15:45:48.285503 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:48.285441    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:45:48.477634 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:48.477601    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"
Jul 27 15:45:51.523566 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:51.523477    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:45:56.504334 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:56.504299    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42"]
Jul 27 15:45:56.508681 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:56.508664    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42"
Jul 27 15:45:56.511488 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:56.511466    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bgfwg\""
Jul 27 15:45:56.515715 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:56.515694    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42"]
Jul 27 15:45:56.682522 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:56.682491    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b72350-6378-40fc-b1fd-d805b9f13ae0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-fcb199-b8g42\" (UID: \"17b72350-6378-40fc-b1fd-d805b9f13ae0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42"
Jul 27 15:45:56.783474 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:56.783401    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b72350-6378-40fc-b1fd-d805b9f13ae0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-fcb199-b8g42\" (UID: \"17b72350-6378-40fc-b1fd-d805b9f13ae0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42"
Jul 27 15:45:56.784011 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:56.783994    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b72350-6378-40fc-b1fd-d805b9f13ae0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-fcb199-b8g42\" (UID: \"17b72350-6378-40fc-b1fd-d805b9f13ae0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42"
Jul 27 15:45:56.821043 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:56.820998    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42"
Jul 27 15:45:56.967264 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:56.967237    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42"]
Jul 27 15:45:56.968908 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:45:56.968874    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17b72350_6378_40fc_b1fd_d805b9f13ae0.slice/crio-19bd50a20e0fb994ab72319e0e1b92bdf4327c41575cdfa37ca09cb47c1917ea WatchSource:0}: Error finding container 19bd50a20e0fb994ab72319e0e1b92bdf4327c41575cdfa37ca09cb47c1917ea: Status 404 returned error can't find the container with id 19bd50a20e0fb994ab72319e0e1b92bdf4327c41575cdfa37ca09cb47c1917ea
Jul 27 15:45:57.582868 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:57.582821    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42" event={"ID":"17b72350-6378-40fc-b1fd-d805b9f13ae0","Type":"ContainerStarted","Data":"72fbbd9d18ca20cc0f14d47761c6164f5d3c13773c853e72ee1a513554e2d513"}
Jul 27 15:45:57.583329 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:57.582883    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42" event={"ID":"17b72350-6378-40fc-b1fd-d805b9f13ae0","Type":"ContainerStarted","Data":"19bd50a20e0fb994ab72319e0e1b92bdf4327c41575cdfa37ca09cb47c1917ea"}
Jul 27 15:45:57.599300 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:57.599243    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42" podStartSLOduration=1.5992252040000001 podStartE2EDuration="1.599225204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:45:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:45:57.598865344 +0000 UTC m=+2429.027540526" watchObservedRunningTime="2026-07-27 15:45:57.599225204 +0000 UTC m=+2429.027900388"
Jul 27 15:45:58.590257 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:58.590221    2548 generic.go:356] "Generic (PLEG): container finished" podID="17b72350-6378-40fc-b1fd-d805b9f13ae0" containerID="72fbbd9d18ca20cc0f14d47761c6164f5d3c13773c853e72ee1a513554e2d513" exitCode=0
Jul 27 15:45:58.590645 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:58.590322    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42" event={"ID":"17b72350-6378-40fc-b1fd-d805b9f13ae0","Type":"ContainerDied","Data":"72fbbd9d18ca20cc0f14d47761c6164f5d3c13773c853e72ee1a513554e2d513"}
Jul 27 15:45:59.732651 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:59.732630    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42"
Jul 27 15:45:59.912106 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:59.912018    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b72350-6378-40fc-b1fd-d805b9f13ae0-maas-api-service-ca\") pod \"17b72350-6378-40fc-b1fd-d805b9f13ae0\" (UID: \"17b72350-6378-40fc-b1fd-d805b9f13ae0\") "
Jul 27 15:45:59.912366 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:45:59.912342    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17b72350-6378-40fc-b1fd-d805b9f13ae0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "17b72350-6378-40fc-b1fd-d805b9f13ae0" (UID: "17b72350-6378-40fc-b1fd-d805b9f13ae0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:46:00.013610 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:00.013578    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b72350-6378-40fc-b1fd-d805b9f13ae0-maas-api-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:46:00.602740 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:00.602704    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42"
Jul 27 15:46:00.602740 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:00.602737    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42" event={"ID":"17b72350-6378-40fc-b1fd-d805b9f13ae0","Type":"ContainerDied","Data":"19bd50a20e0fb994ab72319e0e1b92bdf4327c41575cdfa37ca09cb47c1917ea"}
Jul 27 15:46:00.602946 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:00.602782    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19bd50a20e0fb994ab72319e0e1b92bdf4327c41575cdfa37ca09cb47c1917ea"
Jul 27 15:46:12.163996 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.163960    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"]
Jul 27 15:46:12.164836 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.164785    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92" podUID="6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa" containerName="maas-api" containerID="cri-o://de8540eb5a9fa67c84bda0c1e9367ad7d7977ec4f714c85ae99f5b48cae0a94d" gracePeriod=30
Jul 27 15:46:12.203242 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.203204    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:12.218511 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.218484    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"]
Jul 27 15:46:12.218740 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.218703    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch" podUID="df2a71cc-90a5-45c2-b66b-c674b38baf92" containerName="payload-processing" containerID="cri-o://711346d7af6b1970574f79a278b222239fcd1b9faed8996999fee4bf5de3206b" gracePeriod=30
Jul 27 15:46:12.466246 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.466223    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:46:12.498787 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.498749    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"
Jul 27 15:46:12.527477 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.527449    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-openshift-service-ca\") pod \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\" (UID: \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\") "
Jul 27 15:46:12.527622 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.527500    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-maas-api-tls\") pod \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\" (UID: \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\") "
Jul 27 15:46:12.527622 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.527564    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8b82\" (UniqueName: \"kubernetes.io/projected/df2a71cc-90a5-45c2-b66b-c674b38baf92-kube-api-access-r8b82\") pod \"df2a71cc-90a5-45c2-b66b-c674b38baf92\" (UID: \"df2a71cc-90a5-45c2-b66b-c674b38baf92\") "
Jul 27 15:46:12.527739 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.527630    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kw4pq\" (UniqueName: \"kubernetes.io/projected/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-kube-api-access-kw4pq\") pod \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\" (UID: \"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa\") "
Jul 27 15:46:12.527739 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.527676    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df2a71cc-90a5-45c2-b66b-c674b38baf92-plugins-config-volume\") pod \"df2a71cc-90a5-45c2-b66b-c674b38baf92\" (UID: \"df2a71cc-90a5-45c2-b66b-c674b38baf92\") "
Jul 27 15:46:12.527852 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.527808    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa" (UID: "6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:46:12.528071 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.528051    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:46:12.528171 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.528144    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df2a71cc-90a5-45c2-b66b-c674b38baf92-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "df2a71cc-90a5-45c2-b66b-c674b38baf92" (UID: "df2a71cc-90a5-45c2-b66b-c674b38baf92"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:46:12.529844 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.529824    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa" (UID: "6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:46:12.529925 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.529900    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-kube-api-access-kw4pq" (OuterVolumeSpecName: "kube-api-access-kw4pq") pod "6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa" (UID: "6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa"). InnerVolumeSpecName "kube-api-access-kw4pq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:46:12.530209 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.530191    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df2a71cc-90a5-45c2-b66b-c674b38baf92-kube-api-access-r8b82" (OuterVolumeSpecName: "kube-api-access-r8b82") pod "df2a71cc-90a5-45c2-b66b-c674b38baf92" (UID: "df2a71cc-90a5-45c2-b66b-c674b38baf92"). InnerVolumeSpecName "kube-api-access-r8b82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:46:12.629036 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.628981    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kw4pq\" (UniqueName: \"kubernetes.io/projected/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-kube-api-access-kw4pq\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:46:12.629036 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.629030    2548 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df2a71cc-90a5-45c2-b66b-c674b38baf92-plugins-config-volume\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:46:12.629036 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.629043    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:46:12.629256 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.629055    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8b82\" (UniqueName: \"kubernetes.io/projected/df2a71cc-90a5-45c2-b66b-c674b38baf92-kube-api-access-r8b82\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:46:12.666446 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.666415    2548 generic.go:356] "Generic (PLEG): container finished" podID="6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa" containerID="de8540eb5a9fa67c84bda0c1e9367ad7d7977ec4f714c85ae99f5b48cae0a94d" exitCode=0
Jul 27 15:46:12.666604 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.666456    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"
Jul 27 15:46:12.666604 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.666482    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92" event={"ID":"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa","Type":"ContainerDied","Data":"de8540eb5a9fa67c84bda0c1e9367ad7d7977ec4f714c85ae99f5b48cae0a94d"}
Jul 27 15:46:12.666604 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.666553    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92" event={"ID":"6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa","Type":"ContainerDied","Data":"73e5947f4aee2b923104e9a49caa6913f77562a0f1184d3848bc94955bc9ac98"}
Jul 27 15:46:12.666604 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.666565    2548 scope.go:117] "RemoveContainer" containerID="de8540eb5a9fa67c84bda0c1e9367ad7d7977ec4f714c85ae99f5b48cae0a94d"
Jul 27 15:46:12.667784 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.667757    2548 generic.go:356] "Generic (PLEG): container finished" podID="df2a71cc-90a5-45c2-b66b-c674b38baf92" containerID="711346d7af6b1970574f79a278b222239fcd1b9faed8996999fee4bf5de3206b" exitCode=0
Jul 27 15:46:12.667916 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.667808    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"
Jul 27 15:46:12.667916 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.667810    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch" event={"ID":"df2a71cc-90a5-45c2-b66b-c674b38baf92","Type":"ContainerDied","Data":"711346d7af6b1970574f79a278b222239fcd1b9faed8996999fee4bf5de3206b"}
Jul 27 15:46:12.668122 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.668011    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch" event={"ID":"df2a71cc-90a5-45c2-b66b-c674b38baf92","Type":"ContainerDied","Data":"7c3df8f3d5a0fe474c74b63bcbff8366c52a8000c35f145119be32a308443c52"}
Jul 27 15:46:12.675358 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.675331    2548 scope.go:117] "RemoveContainer" containerID="de8540eb5a9fa67c84bda0c1e9367ad7d7977ec4f714c85ae99f5b48cae0a94d"
Jul 27 15:46:12.675676 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:46:12.675651    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de8540eb5a9fa67c84bda0c1e9367ad7d7977ec4f714c85ae99f5b48cae0a94d\": container with ID starting with de8540eb5a9fa67c84bda0c1e9367ad7d7977ec4f714c85ae99f5b48cae0a94d not found: ID does not exist" containerID="de8540eb5a9fa67c84bda0c1e9367ad7d7977ec4f714c85ae99f5b48cae0a94d"
Jul 27 15:46:12.675755 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.675686    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de8540eb5a9fa67c84bda0c1e9367ad7d7977ec4f714c85ae99f5b48cae0a94d"} err="failed to get container status \"de8540eb5a9fa67c84bda0c1e9367ad7d7977ec4f714c85ae99f5b48cae0a94d\": rpc error: code = NotFound desc = could not find container \"de8540eb5a9fa67c84bda0c1e9367ad7d7977ec4f714c85ae99f5b48cae0a94d\": container with ID starting with de8540eb5a9fa67c84bda0c1e9367ad7d7977ec4f714c85ae99f5b48cae0a94d not found: ID does not exist"
Jul 27 15:46:12.675755 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.675709    2548 scope.go:117] "RemoveContainer" containerID="711346d7af6b1970574f79a278b222239fcd1b9faed8996999fee4bf5de3206b"
Jul 27 15:46:12.683908 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.683889    2548 scope.go:117] "RemoveContainer" containerID="711346d7af6b1970574f79a278b222239fcd1b9faed8996999fee4bf5de3206b"
Jul 27 15:46:12.684165 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:46:12.684145    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"711346d7af6b1970574f79a278b222239fcd1b9faed8996999fee4bf5de3206b\": container with ID starting with 711346d7af6b1970574f79a278b222239fcd1b9faed8996999fee4bf5de3206b not found: ID does not exist" containerID="711346d7af6b1970574f79a278b222239fcd1b9faed8996999fee4bf5de3206b"
Jul 27 15:46:12.684215 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.684174    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711346d7af6b1970574f79a278b222239fcd1b9faed8996999fee4bf5de3206b"} err="failed to get container status \"711346d7af6b1970574f79a278b222239fcd1b9faed8996999fee4bf5de3206b\": rpc error: code = NotFound desc = could not find container \"711346d7af6b1970574f79a278b222239fcd1b9faed8996999fee4bf5de3206b\": container with ID starting with 711346d7af6b1970574f79a278b222239fcd1b9faed8996999fee4bf5de3206b not found: ID does not exist"
Jul 27 15:46:12.692312 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.692288    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"]
Jul 27 15:46:12.699156 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.699130    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fcb199-695946fb8-zqfch"]
Jul 27 15:46:12.709498 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.709476    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"]
Jul 27 15:46:12.718571 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:12.718553    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fcb199-5996c9ffc8-wcm92"]
Jul 27 15:46:13.138591 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:13.138562    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa" path="/var/lib/kubelet/pods/6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa/volumes"
Jul 27 15:46:13.138927 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:13.138913    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df2a71cc-90a5-45c2-b66b-c674b38baf92" path="/var/lib/kubelet/pods/df2a71cc-90a5-45c2-b66b-c674b38baf92/volumes"
Jul 27 15:46:13.478442 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:13.478388    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:16.583928 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:16.583898    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42"]
Jul 27 15:46:16.587090 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:16.587068    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fcb199-b8g42"]
Jul 27 15:46:16.589777 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:16.589752    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:16.965175 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:16.965093    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:17.138227 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:17.138192    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17b72350-6378-40fc-b1fd-d805b9f13ae0" path="/var/lib/kubelet/pods/17b72350-6378-40fc-b1fd-d805b9f13ae0/volumes"
Jul 27 15:46:22.674086 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:22.674052    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:22.761580 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:22.761553    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:22.966016 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:22.965932    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:23.205989 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.205955    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:23.312164 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.312126    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n"]
Jul 27 15:46:23.312504 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.312492    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17b72350-6378-40fc-b1fd-d805b9f13ae0" containerName="revoke-keys"
Jul 27 15:46:23.312581 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.312505    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="17b72350-6378-40fc-b1fd-d805b9f13ae0" containerName="revoke-keys"
Jul 27 15:46:23.312581 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.312520    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df2a71cc-90a5-45c2-b66b-c674b38baf92" containerName="payload-processing"
Jul 27 15:46:23.312581 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.312526    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="df2a71cc-90a5-45c2-b66b-c674b38baf92" containerName="payload-processing"
Jul 27 15:46:23.312581 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.312555    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa" containerName="maas-api"
Jul 27 15:46:23.312581 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.312561    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa" containerName="maas-api"
Jul 27 15:46:23.312753 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.312629    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f6500e9-d9b4-4cc4-a831-83e85e4c8ffa" containerName="maas-api"
Jul 27 15:46:23.312753 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.312637    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="df2a71cc-90a5-45c2-b66b-c674b38baf92" containerName="payload-processing"
Jul 27 15:46:23.312753 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.312647    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="17b72350-6378-40fc-b1fd-d805b9f13ae0" containerName="revoke-keys"
Jul 27 15:46:23.315872 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.315856    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n"
Jul 27 15:46:23.318579 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.318562    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bgfwg\""
Jul 27 15:46:23.323429 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.323407    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n"]
Jul 27 15:46:23.424672 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.424636    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n\" (UID: \"da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n"
Jul 27 15:46:23.525748 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.525717    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n\" (UID: \"da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n"
Jul 27 15:46:23.526356 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.526323    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n\" (UID: \"da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n"
Jul 27 15:46:23.626826 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.626754    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n"
Jul 27 15:46:23.762686 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:23.762663    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n"]
Jul 27 15:46:23.764662 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:46:23.764623    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda3c5aeb_bb58_4d01_b3b9_f159d01ceb5d.slice/crio-0ac3a1f0aa026bd2c856c1ac5e32b50c44d2d0fbcfe8d8cacf5ba0332b0a63fd WatchSource:0}: Error finding container 0ac3a1f0aa026bd2c856c1ac5e32b50c44d2d0fbcfe8d8cacf5ba0332b0a63fd: Status 404 returned error can't find the container with id 0ac3a1f0aa026bd2c856c1ac5e32b50c44d2d0fbcfe8d8cacf5ba0332b0a63fd
Jul 27 15:46:24.743890 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:24.743808    2548 generic.go:356] "Generic (PLEG): container finished" podID="da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d" containerID="41c592a421c8608c7d61a4e9b26c9cfa81a2d829604bb418b511c4f19412fe04" exitCode=0
Jul 27 15:46:24.744030 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:24.743881    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n" event={"ID":"da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d","Type":"ContainerDied","Data":"41c592a421c8608c7d61a4e9b26c9cfa81a2d829604bb418b511c4f19412fe04"}
Jul 27 15:46:24.744030 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:24.743932    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n" event={"ID":"da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d","Type":"ContainerStarted","Data":"0ac3a1f0aa026bd2c856c1ac5e32b50c44d2d0fbcfe8d8cacf5ba0332b0a63fd"}
Jul 27 15:46:25.879026 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:25.879000    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n"
Jul 27 15:46:25.947042 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:25.947011    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d-maas-api-service-ca\") pod \"da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d\" (UID: \"da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d\") "
Jul 27 15:46:25.947377 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:25.947356    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d" (UID: "da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:46:26.048290 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:26.048207    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d-maas-api-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:46:26.755279 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:26.755248    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n" event={"ID":"da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d","Type":"ContainerDied","Data":"0ac3a1f0aa026bd2c856c1ac5e32b50c44d2d0fbcfe8d8cacf5ba0332b0a63fd"}
Jul 27 15:46:26.755279 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:26.755276    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n"
Jul 27 15:46:26.755484 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:26.755293    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ac3a1f0aa026bd2c856c1ac5e32b50c44d2d0fbcfe8d8cacf5ba0332b0a63fd"
Jul 27 15:46:29.423834 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:29.423801    2548 scope.go:117] "RemoveContainer" containerID="b3eb6ae9c1c02349273f7f380b83a1884b77909fc7aa27a8e2bfc9cef9973e34"
Jul 27 15:46:38.879810 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:38.879773    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"]
Jul 27 15:46:38.880295 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:38.880043    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q" podUID="be519e10-8139-4a09-8f62-d7fcf70a6287" containerName="maas-api" containerID="cri-o://afeb8accb04e5762670db7cc23625895ca8ed2f52e9fa922b24d9247ea82a2c6" gracePeriod=30
Jul 27 15:46:38.910740 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:38.910711    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:39.011372 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.011338    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:39.112487 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.112463    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:39.132246 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.132198    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:46:39.267369 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.267341    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be519e10-8139-4a09-8f62-d7fcf70a6287-openshift-service-ca\") pod \"be519e10-8139-4a09-8f62-d7fcf70a6287\" (UID: \"be519e10-8139-4a09-8f62-d7fcf70a6287\") "
Jul 27 15:46:39.267528 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.267461    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be519e10-8139-4a09-8f62-d7fcf70a6287-maas-api-tls\") pod \"be519e10-8139-4a09-8f62-d7fcf70a6287\" (UID: \"be519e10-8139-4a09-8f62-d7fcf70a6287\") "
Jul 27 15:46:39.267528 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.267500    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dlvn\" (UniqueName: \"kubernetes.io/projected/be519e10-8139-4a09-8f62-d7fcf70a6287-kube-api-access-7dlvn\") pod \"be519e10-8139-4a09-8f62-d7fcf70a6287\" (UID: \"be519e10-8139-4a09-8f62-d7fcf70a6287\") "
Jul 27 15:46:39.267759 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.267737    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be519e10-8139-4a09-8f62-d7fcf70a6287-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "be519e10-8139-4a09-8f62-d7fcf70a6287" (UID: "be519e10-8139-4a09-8f62-d7fcf70a6287"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:46:39.269715 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.269687    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be519e10-8139-4a09-8f62-d7fcf70a6287-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "be519e10-8139-4a09-8f62-d7fcf70a6287" (UID: "be519e10-8139-4a09-8f62-d7fcf70a6287"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:46:39.269810 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.269738    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be519e10-8139-4a09-8f62-d7fcf70a6287-kube-api-access-7dlvn" (OuterVolumeSpecName: "kube-api-access-7dlvn") pod "be519e10-8139-4a09-8f62-d7fcf70a6287" (UID: "be519e10-8139-4a09-8f62-d7fcf70a6287"). InnerVolumeSpecName "kube-api-access-7dlvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:46:39.368955 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.368929    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be519e10-8139-4a09-8f62-d7fcf70a6287-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:46:39.368955 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.368953    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be519e10-8139-4a09-8f62-d7fcf70a6287-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:46:39.368955 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.368962    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dlvn\" (UniqueName: \"kubernetes.io/projected/be519e10-8139-4a09-8f62-d7fcf70a6287-kube-api-access-7dlvn\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:46:39.823682 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.823645    2548 generic.go:356] "Generic (PLEG): container finished" podID="be519e10-8139-4a09-8f62-d7fcf70a6287" containerID="afeb8accb04e5762670db7cc23625895ca8ed2f52e9fa922b24d9247ea82a2c6" exitCode=0
Jul 27 15:46:39.823682 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.823684    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"
Jul 27 15:46:39.823918 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.823735    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q" event={"ID":"be519e10-8139-4a09-8f62-d7fcf70a6287","Type":"ContainerDied","Data":"afeb8accb04e5762670db7cc23625895ca8ed2f52e9fa922b24d9247ea82a2c6"}
Jul 27 15:46:39.823918 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.823791    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q" event={"ID":"be519e10-8139-4a09-8f62-d7fcf70a6287","Type":"ContainerDied","Data":"6166268b1a2500a9ffb14b09d637f2ed1fdbab58dffc1d742df66554b73c4ea7"}
Jul 27 15:46:39.823918 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.823815    2548 scope.go:117] "RemoveContainer" containerID="afeb8accb04e5762670db7cc23625895ca8ed2f52e9fa922b24d9247ea82a2c6"
Jul 27 15:46:39.839498 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.839466    2548 scope.go:117] "RemoveContainer" containerID="afeb8accb04e5762670db7cc23625895ca8ed2f52e9fa922b24d9247ea82a2c6"
Jul 27 15:46:39.839851 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:46:39.839828    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afeb8accb04e5762670db7cc23625895ca8ed2f52e9fa922b24d9247ea82a2c6\": container with ID starting with afeb8accb04e5762670db7cc23625895ca8ed2f52e9fa922b24d9247ea82a2c6 not found: ID does not exist" containerID="afeb8accb04e5762670db7cc23625895ca8ed2f52e9fa922b24d9247ea82a2c6"
Jul 27 15:46:39.839934 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.839858    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afeb8accb04e5762670db7cc23625895ca8ed2f52e9fa922b24d9247ea82a2c6"} err="failed to get container status \"afeb8accb04e5762670db7cc23625895ca8ed2f52e9fa922b24d9247ea82a2c6\": rpc error: code = NotFound desc = could not find container \"afeb8accb04e5762670db7cc23625895ca8ed2f52e9fa922b24d9247ea82a2c6\": container with ID starting with afeb8accb04e5762670db7cc23625895ca8ed2f52e9fa922b24d9247ea82a2c6 not found: ID does not exist"
Jul 27 15:46:39.850783 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.850760    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"]
Jul 27 15:46:39.855432 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:39.855410    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fbf2d4-6487ffd757-ctl5q"]
Jul 27 15:46:41.138876 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:41.138841    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be519e10-8139-4a09-8f62-d7fcf70a6287" path="/var/lib/kubelet/pods/be519e10-8139-4a09-8f62-d7fcf70a6287/volumes"
Jul 27 15:46:43.390923 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:43.390890    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n"]
Jul 27 15:46:43.392845 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:43.392820    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fbf2d4-ht66n"]
Jul 27 15:46:43.393083 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:43.393064    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:43.784333 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:43.784302    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:45.138417 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:45.138386    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d" path="/var/lib/kubelet/pods/da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d/volumes"
Jul 27 15:46:49.652398 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:49.652369    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:50.248492 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:50.248461    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:50.313801 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:50.313721    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:50.512805 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:50.512729    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:50.839853 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:50.839771    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:51.492874 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:51.492833    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:54.228699 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.228660    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:54.300757 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.300719    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"]
Jul 27 15:46:54.301339 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.301314    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be519e10-8139-4a09-8f62-d7fcf70a6287" containerName="maas-api"
Jul 27 15:46:54.301339 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.301340    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="be519e10-8139-4a09-8f62-d7fcf70a6287" containerName="maas-api"
Jul 27 15:46:54.301339 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.301357    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d" containerName="revoke-keys"
Jul 27 15:46:54.301630 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.301366    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d" containerName="revoke-keys"
Jul 27 15:46:54.301630 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.301460    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="da3c5aeb-bb58-4d01-b3b9-f159d01ceb5d" containerName="revoke-keys"
Jul 27 15:46:54.301630 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.301477    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="be519e10-8139-4a09-8f62-d7fcf70a6287" containerName="maas-api"
Jul 27 15:46:54.305911 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.305894    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 15:46:54.308581 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.308563    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-f58a29-serving-cert\""
Jul 27 15:46:54.314376 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.314350    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"]
Jul 27 15:46:54.321407 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.321388    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:54.401898 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.401869    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2fv5\" (UniqueName: \"kubernetes.io/projected/33f33f31-96fe-4603-adac-7dbebcd91e1c-kube-api-access-m2fv5\") pod \"maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8\" (UID: \"33f33f31-96fe-4603-adac-7dbebcd91e1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 15:46:54.401898 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.401903    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f33f31-96fe-4603-adac-7dbebcd91e1c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8\" (UID: \"33f33f31-96fe-4603-adac-7dbebcd91e1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 15:46:54.402149 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.401928    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f33f31-96fe-4603-adac-7dbebcd91e1c-maas-api-tls\") pod \"maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8\" (UID: \"33f33f31-96fe-4603-adac-7dbebcd91e1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 15:46:54.460014 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.459981    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:54.503387 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.503319    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2fv5\" (UniqueName: \"kubernetes.io/projected/33f33f31-96fe-4603-adac-7dbebcd91e1c-kube-api-access-m2fv5\") pod \"maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8\" (UID: \"33f33f31-96fe-4603-adac-7dbebcd91e1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 15:46:54.503387 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.503349    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f33f31-96fe-4603-adac-7dbebcd91e1c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8\" (UID: \"33f33f31-96fe-4603-adac-7dbebcd91e1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 15:46:54.503387 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.503385    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f33f31-96fe-4603-adac-7dbebcd91e1c-maas-api-tls\") pod \"maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8\" (UID: \"33f33f31-96fe-4603-adac-7dbebcd91e1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 15:46:54.503651 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:46:54.503493    2548 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-serving-cert: secret "maas-api-e2e-shared-a-f58a29-serving-cert" not found
Jul 27 15:46:54.503651 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:46:54.503567    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33f33f31-96fe-4603-adac-7dbebcd91e1c-maas-api-tls podName:33f33f31-96fe-4603-adac-7dbebcd91e1c nodeName:}" failed. No retries permitted until 2026-07-27 15:46:55.003527498 +0000 UTC m=+2486.432202658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/33f33f31-96fe-4603-adac-7dbebcd91e1c-maas-api-tls") pod "maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8" (UID: "33f33f31-96fe-4603-adac-7dbebcd91e1c") : secret "maas-api-e2e-shared-a-f58a29-serving-cert" not found
Jul 27 15:46:54.503948 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.503929    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f33f31-96fe-4603-adac-7dbebcd91e1c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8\" (UID: \"33f33f31-96fe-4603-adac-7dbebcd91e1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 15:46:54.514177 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.514157    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2fv5\" (UniqueName: \"kubernetes.io/projected/33f33f31-96fe-4603-adac-7dbebcd91e1c-kube-api-access-m2fv5\") pod \"maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8\" (UID: \"33f33f31-96fe-4603-adac-7dbebcd91e1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 15:46:54.828950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.828836    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:54.919230 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:54.919177    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:55.008263 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:55.008235    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f33f31-96fe-4603-adac-7dbebcd91e1c-maas-api-tls\") pod \"maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8\" (UID: \"33f33f31-96fe-4603-adac-7dbebcd91e1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 15:46:55.010806 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:55.010783    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f33f31-96fe-4603-adac-7dbebcd91e1c-maas-api-tls\") pod \"maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8\" (UID: \"33f33f31-96fe-4603-adac-7dbebcd91e1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 15:46:55.217763 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:55.217679    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 15:46:55.364638 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:55.364609    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"]
Jul 27 15:46:55.366277 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:46:55.366251    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33f33f31_96fe_4603_adac_7dbebcd91e1c.slice/crio-d97eaa95f6ba4891fa3ab18645f04366d012664d08d4294c876394cfa05b3faa WatchSource:0}: Error finding container d97eaa95f6ba4891fa3ab18645f04366d012664d08d4294c876394cfa05b3faa: Status 404 returned error can't find the container with id d97eaa95f6ba4891fa3ab18645f04366d012664d08d4294c876394cfa05b3faa
Jul 27 15:46:55.918640 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:55.918594    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8" event={"ID":"33f33f31-96fe-4603-adac-7dbebcd91e1c","Type":"ContainerStarted","Data":"a4c0d4d955da49006534da06952eb30618c7fb78c68dd047c3e44b79271ddd74"}
Jul 27 15:46:55.918826 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:55.918661    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8" event={"ID":"33f33f31-96fe-4603-adac-7dbebcd91e1c","Type":"ContainerStarted","Data":"d97eaa95f6ba4891fa3ab18645f04366d012664d08d4294c876394cfa05b3faa"}
Jul 27 15:46:55.918826 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:55.918689    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 15:46:55.939133 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:55.939087    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8" podStartSLOduration=1.567210862 podStartE2EDuration="1.939072133s" totalImagesPullingTime="371.861271ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:46:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:46:55.937927301 +0000 UTC m=+2487.366602483" watchObservedRunningTime="2026-07-27 15:46:55.939072133 +0000 UTC m=+2487.367747314"
Jul 27 15:46:58.802411 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:58.802377    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:59.166413 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:59.166330    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:59.483408 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:59.483377    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:46:59.718692 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:46:59.718663    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:47:01.928609 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:01.928578    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 15:47:02.241504 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.241473    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:47:02.310594 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.310567    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"]
Jul 27 15:47:02.314415 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.314393    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 15:47:02.317220 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.317197    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-052d16-serving-cert\""
Jul 27 15:47:02.326817 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.326796    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"]
Jul 27 15:47:02.331400 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.331380    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:47:02.381154 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.381119    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85576123-98c8-413d-88b7-d386d850d444-openshift-service-ca\") pod \"maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt\" (UID: \"85576123-98c8-413d-88b7-d386d850d444\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 15:47:02.381315 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.381178    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85576123-98c8-413d-88b7-d386d850d444-maas-api-tls\") pod \"maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt\" (UID: \"85576123-98c8-413d-88b7-d386d850d444\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 15:47:02.381315 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.381283    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n65mn\" (UniqueName: \"kubernetes.io/projected/85576123-98c8-413d-88b7-d386d850d444-kube-api-access-n65mn\") pod \"maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt\" (UID: \"85576123-98c8-413d-88b7-d386d850d444\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 15:47:02.481360 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.481331    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:47:02.481780 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.481761    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n65mn\" (UniqueName: \"kubernetes.io/projected/85576123-98c8-413d-88b7-d386d850d444-kube-api-access-n65mn\") pod \"maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt\" (UID: \"85576123-98c8-413d-88b7-d386d850d444\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 15:47:02.481831 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.481815    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85576123-98c8-413d-88b7-d386d850d444-openshift-service-ca\") pod \"maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt\" (UID: \"85576123-98c8-413d-88b7-d386d850d444\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 15:47:02.481872 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.481842    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85576123-98c8-413d-88b7-d386d850d444-maas-api-tls\") pod \"maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt\" (UID: \"85576123-98c8-413d-88b7-d386d850d444\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 15:47:02.481954 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:47:02.481940    2548 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-serving-cert: secret "maas-api-e2e-shared-b-052d16-serving-cert" not found
Jul 27 15:47:02.482008 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:47:02.481999    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85576123-98c8-413d-88b7-d386d850d444-maas-api-tls podName:85576123-98c8-413d-88b7-d386d850d444 nodeName:}" failed. No retries permitted until 2026-07-27 15:47:02.98198487 +0000 UTC m=+2494.410660032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/85576123-98c8-413d-88b7-d386d850d444-maas-api-tls") pod "maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt" (UID: "85576123-98c8-413d-88b7-d386d850d444") : secret "maas-api-e2e-shared-b-052d16-serving-cert" not found
Jul 27 15:47:02.482408 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.482391    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85576123-98c8-413d-88b7-d386d850d444-openshift-service-ca\") pod \"maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt\" (UID: \"85576123-98c8-413d-88b7-d386d850d444\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 15:47:02.494702 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.494641    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n65mn\" (UniqueName: \"kubernetes.io/projected/85576123-98c8-413d-88b7-d386d850d444-kube-api-access-n65mn\") pod \"maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt\" (UID: \"85576123-98c8-413d-88b7-d386d850d444\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 15:47:02.851493 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.851395    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:47:02.949856 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.949826    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:47:02.987498 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.987461    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85576123-98c8-413d-88b7-d386d850d444-maas-api-tls\") pod \"maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt\" (UID: \"85576123-98c8-413d-88b7-d386d850d444\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 15:47:02.990159 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:02.990130    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85576123-98c8-413d-88b7-d386d850d444-maas-api-tls\") pod \"maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt\" (UID: \"85576123-98c8-413d-88b7-d386d850d444\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 15:47:03.044353 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:03.044327    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:47:03.225435 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:03.225342    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 15:47:03.598861 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:03.598836    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"]
Jul 27 15:47:03.600772 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:47:03.600740    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85576123_98c8_413d_88b7_d386d850d444.slice/crio-f354cf38f1030ac089560d9cbbd63a0a0b1d99105ae1dc4733084bc7bf8798b5 WatchSource:0}: Error finding container f354cf38f1030ac089560d9cbbd63a0a0b1d99105ae1dc4733084bc7bf8798b5: Status 404 returned error can't find the container with id f354cf38f1030ac089560d9cbbd63a0a0b1d99105ae1dc4733084bc7bf8798b5
Jul 27 15:47:03.961138 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:03.961030    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt" event={"ID":"85576123-98c8-413d-88b7-d386d850d444","Type":"ContainerStarted","Data":"f354cf38f1030ac089560d9cbbd63a0a0b1d99105ae1dc4733084bc7bf8798b5"}
Jul 27 15:47:04.970552 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:04.970446    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt" event={"ID":"85576123-98c8-413d-88b7-d386d850d444","Type":"ContainerStarted","Data":"fc2e11a39e93f8a6cc4da6a985938a38036a97f584ee1504e1fb7f3d6da75a64"}
Jul 27 15:47:04.970989 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:04.970586    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 15:47:04.990433 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:04.990377    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt" podStartSLOduration=1.965362074 podStartE2EDuration="2.990358802s" totalImagesPullingTime="1.024996728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:47:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:47:04.989019884 +0000 UTC m=+2496.417695048" watchObservedRunningTime="2026-07-27 15:47:04.990358802 +0000 UTC m=+2496.419033985"
Jul 27 15:47:06.802411 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:06.802378    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:47:10.980976 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:10.980945    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 15:47:15.273439 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:15.273403    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:47:29.454309 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:29.454234    2548 scope.go:117] "RemoveContainer" containerID="92bbfb8d352a2ea20492f509e083a4e9fba36a447c317a9c4d28c7931283b2b4"
Jul 27 15:47:38.404146 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:47:38.404114    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:48:03.980006 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:48:03.979956    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:48:12.046479 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:48:12.046443    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:48:32.390334 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:48:32.390292    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:49:10.949501 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:49:10.949397    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:49:11.049256 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:49:11.049220    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:50:10.767280 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:50:10.767248    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:51:29.499422 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:51:29.499386    2548 scope.go:117] "RemoveContainer" containerID="c636c55755f6aebac2a0d34dab52921e8605cbfb7c1c6a4d21be5a401ad62d42"
Jul 27 15:51:29.508384 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:51:29.508363    2548 scope.go:117] "RemoveContainer" containerID="be59e99e82d6eea0c82db979c613a8ed91c2b36c37e6a5e30dbaab22b8054edb"
Jul 27 15:51:41.018238 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:51:41.018202    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:52:19.678961 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:52:19.678884    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:52:20.574863 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:52:20.574831    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:52:29.529643 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:52:29.529610    2548 scope.go:117] "RemoveContainer" containerID="41c592a421c8608c7d61a4e9b26c9cfa81a2d829604bb418b511c4f19412fe04"
Jul 27 15:52:29.541374 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:52:29.541345    2548 scope.go:117] "RemoveContainer" containerID="72fbbd9d18ca20cc0f14d47761c6164f5d3c13773c853e72ee1a513554e2d513"
Jul 27 15:52:35.844842 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:52:35.844784    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:52:59.508000 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:52:59.507969    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:54:00.511475 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:54:00.511440    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:54:01.704839 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:54:01.704801    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:54:11.125952 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:54:11.125909    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:54:44.316188 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:54:44.316156    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:55:37.599059 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:55:37.599016    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:55:42.415922 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:55:42.415889    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:03.881450 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:03.881419    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:04.781810 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:04.781780    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:24.876372 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:24.876339    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:25.156001 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:25.155917    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:25.295856 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:25.295828    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:28.118112 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.118079    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:28.196259 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.196211    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:28.274055 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.274026    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"]
Jul 27 15:56:28.277442 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.277425    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:56:28.280042 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.280021    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-c4dbeb-serving-cert\""
Jul 27 15:56:28.290322 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.290301    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"]
Jul 27 15:56:28.326491 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.326462    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:28.333852 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.333829    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d901721e-8301-441f-9a5c-cb9f627969ea-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q\" (UID: \"d901721e-8301-441f-9a5c-cb9f627969ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:56:28.333960 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.333935    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d901721e-8301-441f-9a5c-cb9f627969ea-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q\" (UID: \"d901721e-8301-441f-9a5c-cb9f627969ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:56:28.334015 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.333970    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk7wc\" (UniqueName: \"kubernetes.io/projected/d901721e-8301-441f-9a5c-cb9f627969ea-kube-api-access-wk7wc\") pod \"maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q\" (UID: \"d901721e-8301-441f-9a5c-cb9f627969ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:56:28.435396 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.435310    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d901721e-8301-441f-9a5c-cb9f627969ea-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q\" (UID: \"d901721e-8301-441f-9a5c-cb9f627969ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:56:28.435396 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.435360    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk7wc\" (UniqueName: \"kubernetes.io/projected/d901721e-8301-441f-9a5c-cb9f627969ea-kube-api-access-wk7wc\") pod \"maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q\" (UID: \"d901721e-8301-441f-9a5c-cb9f627969ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:56:28.435656 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.435430    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d901721e-8301-441f-9a5c-cb9f627969ea-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q\" (UID: \"d901721e-8301-441f-9a5c-cb9f627969ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:56:28.435656 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:56:28.435454    2548 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-serving-cert: secret "maas-api-e2e-ipp-a-c4dbeb-serving-cert" not found
Jul 27 15:56:28.435656 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:56:28.435558    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d901721e-8301-441f-9a5c-cb9f627969ea-maas-api-tls podName:d901721e-8301-441f-9a5c-cb9f627969ea nodeName:}" failed. No retries permitted until 2026-07-27 15:56:28.935512212 +0000 UTC m=+3060.364187388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d901721e-8301-441f-9a5c-cb9f627969ea-maas-api-tls") pod "maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q" (UID: "d901721e-8301-441f-9a5c-cb9f627969ea") : secret "maas-api-e2e-ipp-a-c4dbeb-serving-cert" not found
Jul 27 15:56:28.436123 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.436102    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d901721e-8301-441f-9a5c-cb9f627969ea-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q\" (UID: \"d901721e-8301-441f-9a5c-cb9f627969ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:56:28.448609 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.446618    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk7wc\" (UniqueName: \"kubernetes.io/projected/d901721e-8301-441f-9a5c-cb9f627969ea-kube-api-access-wk7wc\") pod \"maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q\" (UID: \"d901721e-8301-441f-9a5c-cb9f627969ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:56:28.692886 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.692798    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:28.883302 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.883261    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:28.939822 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.939784    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d901721e-8301-441f-9a5c-cb9f627969ea-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q\" (UID: \"d901721e-8301-441f-9a5c-cb9f627969ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:56:28.942391 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.942370    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d901721e-8301-441f-9a5c-cb9f627969ea-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q\" (UID: \"d901721e-8301-441f-9a5c-cb9f627969ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:56:28.977664 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.977631    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:28.979273 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.979254    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"]
Jul 27 15:56:28.982728 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.982714    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"
Jul 27 15:56:28.985871 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.985843    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-c4dbeb\""
Jul 27 15:56:28.994018 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:28.993995    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"]
Jul 27 15:56:29.009758 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.009731    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"]
Jul 27 15:56:29.013593 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.013575    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"
Jul 27 15:56:29.016332 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.016310    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-c4dbeb-dockercfg-9s6rd\""
Jul 27 15:56:29.024642 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.024619    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"]
Jul 27 15:56:29.041289 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.041267    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45\" (UID: \"c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"
Jul 27 15:56:29.041401 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.041326    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rlr8\" (UniqueName: \"kubernetes.io/projected/c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792-kube-api-access-5rlr8\") pod \"payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45\" (UID: \"c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"
Jul 27 15:56:29.142656 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.142168    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45\" (UID: \"c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"
Jul 27 15:56:29.142656 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.142250    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31f31e42-c018-494e-b03b-6017768547c2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk\" (UID: \"31f31e42-c018-494e-b03b-6017768547c2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"
Jul 27 15:56:29.142656 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.142286    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rlr8\" (UniqueName: \"kubernetes.io/projected/c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792-kube-api-access-5rlr8\") pod \"payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45\" (UID: \"c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"
Jul 27 15:56:29.142656 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.142379    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxjdz\" (UniqueName: \"kubernetes.io/projected/31f31e42-c018-494e-b03b-6017768547c2-kube-api-access-bxjdz\") pod \"payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk\" (UID: \"31f31e42-c018-494e-b03b-6017768547c2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"
Jul 27 15:56:29.145307 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.145288    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-c4dbeb\""
Jul 27 15:56:29.152059 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.152033    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rlr8\" (UniqueName: \"kubernetes.io/projected/c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792-kube-api-access-5rlr8\") pod \"payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45\" (UID: \"c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"
Jul 27 15:56:29.152999 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.152977    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45\" (UID: \"c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"
Jul 27 15:56:29.189187 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.189161    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:56:29.244148 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.243661    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31f31e42-c018-494e-b03b-6017768547c2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk\" (UID: \"31f31e42-c018-494e-b03b-6017768547c2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"
Jul 27 15:56:29.244148 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.243725    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxjdz\" (UniqueName: \"kubernetes.io/projected/31f31e42-c018-494e-b03b-6017768547c2-kube-api-access-bxjdz\") pod \"payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk\" (UID: \"31f31e42-c018-494e-b03b-6017768547c2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"
Jul 27 15:56:29.245019 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.244980    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31f31e42-c018-494e-b03b-6017768547c2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk\" (UID: \"31f31e42-c018-494e-b03b-6017768547c2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"
Jul 27 15:56:29.254655 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.253930    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxjdz\" (UniqueName: \"kubernetes.io/projected/31f31e42-c018-494e-b03b-6017768547c2-kube-api-access-bxjdz\") pod \"payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk\" (UID: \"31f31e42-c018-494e-b03b-6017768547c2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"
Jul 27 15:56:29.295798 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.295763    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"
Jul 27 15:56:29.325129 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.324981    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"
Jul 27 15:56:29.333297 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.333265    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"]
Jul 27 15:56:29.338607 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:56:29.338576    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd901721e_8301_441f_9a5c_cb9f627969ea.slice/crio-725a27be203d2f48f4b30e35673b1f3c091fb17e3bab6cea62773faed43c2195 WatchSource:0}: Error finding container 725a27be203d2f48f4b30e35673b1f3c091fb17e3bab6cea62773faed43c2195: Status 404 returned error can't find the container with id 725a27be203d2f48f4b30e35673b1f3c091fb17e3bab6cea62773faed43c2195
Jul 27 15:56:29.339962 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.339943    2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 15:56:29.461139 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.461040    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"]
Jul 27 15:56:29.465105 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:56:29.465073    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6482f1c_3c14_4dcf_bfdd_b3a3a45f0792.slice/crio-d2b87bf06f16b3fb404f53f833a6adc3b7d6e9f8579d478d2aefa3bfc6ff05ac WatchSource:0}: Error finding container d2b87bf06f16b3fb404f53f833a6adc3b7d6e9f8579d478d2aefa3bfc6ff05ac: Status 404 returned error can't find the container with id d2b87bf06f16b3fb404f53f833a6adc3b7d6e9f8579d478d2aefa3bfc6ff05ac
Jul 27 15:56:29.693307 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.693128    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"]
Jul 27 15:56:29.696034 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:56:29.696006    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31f31e42_c018_494e_b03b_6017768547c2.slice/crio-4ff42e3832bdbfd14ddbfcb2eba020d0457461e8d1b91e54b2203d7dcc1ea534 WatchSource:0}: Error finding container 4ff42e3832bdbfd14ddbfcb2eba020d0457461e8d1b91e54b2203d7dcc1ea534: Status 404 returned error can't find the container with id 4ff42e3832bdbfd14ddbfcb2eba020d0457461e8d1b91e54b2203d7dcc1ea534
Jul 27 15:56:29.795345 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.795308    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45" event={"ID":"c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792","Type":"ContainerStarted","Data":"4174e229b161f1c96391d4f5a9dc8035b99629601e61b2b43cb7198c0ab6e9bb"}
Jul 27 15:56:29.795520 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.795369    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45" event={"ID":"c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792","Type":"ContainerStarted","Data":"d2b87bf06f16b3fb404f53f833a6adc3b7d6e9f8579d478d2aefa3bfc6ff05ac"}
Jul 27 15:56:29.795520 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.795496    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"
Jul 27 15:56:29.796684 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.796657    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q" event={"ID":"d901721e-8301-441f-9a5c-cb9f627969ea","Type":"ContainerStarted","Data":"725a27be203d2f48f4b30e35673b1f3c091fb17e3bab6cea62773faed43c2195"}
Jul 27 15:56:29.798338 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.798316    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk" event={"ID":"31f31e42-c018-494e-b03b-6017768547c2","Type":"ContainerStarted","Data":"376ff73aa8d021810060380a13b576a78eb4a07d71efbe95424d61a82b8cdb02"}
Jul 27 15:56:29.798445 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.798358    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk" event={"ID":"31f31e42-c018-494e-b03b-6017768547c2","Type":"ContainerStarted","Data":"4ff42e3832bdbfd14ddbfcb2eba020d0457461e8d1b91e54b2203d7dcc1ea534"}
Jul 27 15:56:29.798445 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.798386    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"
Jul 27 15:56:29.816498 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.816447    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45" podStartSLOduration=1.81643004 podStartE2EDuration="1.81643004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:56:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:56:29.816084055 +0000 UTC m=+3061.244759240" watchObservedRunningTime="2026-07-27 15:56:29.81643004 +0000 UTC m=+3061.245105224"
Jul 27 15:56:29.831440 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:29.831393    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk" podStartSLOduration=1.831377678 podStartE2EDuration="1.831377678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:56:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:56:29.831090663 +0000 UTC m=+3061.259765845" watchObservedRunningTime="2026-07-27 15:56:29.831377678 +0000 UTC m=+3061.260052860"
Jul 27 15:56:30.807116 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:30.807077    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q" event={"ID":"d901721e-8301-441f-9a5c-cb9f627969ea","Type":"ContainerStarted","Data":"1351c1f50fc0e093b9f18a71e318647ec49031dea17f833a532a76e44cd8e7e1"}
Jul 27 15:56:30.807617 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:30.807469    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:56:30.833162 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:30.833114    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q" podStartSLOduration=2.206229718 podStartE2EDuration="2.833098033s" totalImagesPullingTime="626.868315ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:56:30.831077826 +0000 UTC m=+3062.259753011" watchObservedRunningTime="2026-07-27 15:56:30.833098033 +0000 UTC m=+3062.261773254"
Jul 27 15:56:32.661422 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:32.661388    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:37.822526 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:37.822495    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:56:40.809647 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:40.809615    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"
Jul 27 15:56:40.810012 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:40.809775    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"
Jul 27 15:56:43.583478 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:43.583435    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:43.900879 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:43.900805    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:44.266383 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:44.266356    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:47.254347 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.254305    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:47.335011 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.334976    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:47.361455 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.361420    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"]
Jul 27 15:56:47.365701 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.365679    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 15:56:47.368949 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.368927    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-809b00-serving-cert\""
Jul 27 15:56:47.378934 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.378908    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"]
Jul 27 15:56:47.432876 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.432839    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:47.511403 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.511316    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-maas-api-tls\") pod \"maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk\" (UID: \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 15:56:47.511403 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.511367    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjqfj\" (UniqueName: \"kubernetes.io/projected/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-kube-api-access-qjqfj\") pod \"maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk\" (UID: \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 15:56:47.511403 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.511401    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk\" (UID: \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 15:56:47.612256 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.612214    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-maas-api-tls\") pod \"maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk\" (UID: \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 15:56:47.612446 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.612276    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjqfj\" (UniqueName: \"kubernetes.io/projected/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-kube-api-access-qjqfj\") pod \"maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk\" (UID: \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 15:56:47.612446 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:56:47.612360    2548 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-serving-cert: secret "maas-api-e2e-ipp-b-809b00-serving-cert" not found
Jul 27 15:56:47.612446 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.612434    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk\" (UID: \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 15:56:47.612671 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:56:47.612465    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-maas-api-tls podName:bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442 nodeName:}" failed. No retries permitted until 2026-07-27 15:56:48.112448371 +0000 UTC m=+3079.541123532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-maas-api-tls") pod "maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk" (UID: "bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442") : secret "maas-api-e2e-ipp-b-809b00-serving-cert" not found
Jul 27 15:56:47.613029 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.613001    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk\" (UID: \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 15:56:47.624192 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:47.624170    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjqfj\" (UniqueName: \"kubernetes.io/projected/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-kube-api-access-qjqfj\") pod \"maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk\" (UID: \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 15:56:48.007061 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:48.007031    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:48.101675 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:48.101642    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:48.117942 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:48.117907    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-maas-api-tls\") pod \"maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk\" (UID: \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 15:56:48.120919 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:48.120893    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-maas-api-tls\") pod \"maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk\" (UID: \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 15:56:48.214381 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:48.214334    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:48.281067 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:48.280986    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 15:56:48.448152 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:48.448117    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"]
Jul 27 15:56:48.450293 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:56:48.450269    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda50b2a_8bcd_46d3_a4ba_2d17fcbc6442.slice/crio-6e222c7a4795bd98a9ee27b7012eed81434b492351c2f37c925dc315c7f21c1c WatchSource:0}: Error finding container 6e222c7a4795bd98a9ee27b7012eed81434b492351c2f37c925dc315c7f21c1c: Status 404 returned error can't find the container with id 6e222c7a4795bd98a9ee27b7012eed81434b492351c2f37c925dc315c7f21c1c
Jul 27 15:56:48.903615 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:48.903581    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk" event={"ID":"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442","Type":"ContainerStarted","Data":"f492fbbfdc974a9d23dde59a18a1709641836732a201cbefea6a00380687a37f"}
Jul 27 15:56:48.903798 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:48.903650    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 15:56:48.903798 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:48.903678    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk" event={"ID":"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442","Type":"ContainerStarted","Data":"6e222c7a4795bd98a9ee27b7012eed81434b492351c2f37c925dc315c7f21c1c"}
Jul 27 15:56:48.922809 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:48.922758    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk" podStartSLOduration=1.604829789 podStartE2EDuration="1.922740908s" totalImagesPullingTime="317.911119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:56:48.921789781 +0000 UTC m=+3080.350464966" watchObservedRunningTime="2026-07-27 15:56:48.922740908 +0000 UTC m=+3080.351416092"
Jul 27 15:56:51.700847 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:51.700811    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:56:54.915028 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:56:54.915002    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 15:57:15.104290 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:57:15.104255    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:57:15.212996 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:57:15.212967    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:57:20.959690 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:57:20.959656    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:57:21.468129 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:57:21.468096    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:57:51.185374 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:57:51.185298    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:57:51.287413 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:57:51.287380    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:04.182528 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:04.182496    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk_31f31e42-c018-494e-b03b-6017768547c2/payload-processing/0.log"
Jul 27 15:58:04.732950 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:04.732913    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:05.071071 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:05.070991    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:06.085632 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:06.085600    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:07.752042 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:07.752000    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:07.853864 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:07.853835    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:09.055502 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.055474    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"]
Jul 27 15:58:09.059004 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.058985    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:09.061765 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.061740    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-4cd197-serving-cert\""
Jul 27 15:58:09.069290 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.069263    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"]
Jul 27 15:58:09.107519 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.107487    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6pc7\" (UniqueName: \"kubernetes.io/projected/72bf5b60-00c5-4a80-9583-9dc1819afd99-kube-api-access-f6pc7\") pod \"maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g\" (UID: \"72bf5b60-00c5-4a80-9583-9dc1819afd99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:09.107686 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.107563    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72bf5b60-00c5-4a80-9583-9dc1819afd99-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g\" (UID: \"72bf5b60-00c5-4a80-9583-9dc1819afd99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:09.107686 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.107583    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72bf5b60-00c5-4a80-9583-9dc1819afd99-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g\" (UID: \"72bf5b60-00c5-4a80-9583-9dc1819afd99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:09.208328 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.208291    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6pc7\" (UniqueName: \"kubernetes.io/projected/72bf5b60-00c5-4a80-9583-9dc1819afd99-kube-api-access-f6pc7\") pod \"maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g\" (UID: \"72bf5b60-00c5-4a80-9583-9dc1819afd99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:09.208516 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.208357    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72bf5b60-00c5-4a80-9583-9dc1819afd99-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g\" (UID: \"72bf5b60-00c5-4a80-9583-9dc1819afd99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:09.208516 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.208386    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72bf5b60-00c5-4a80-9583-9dc1819afd99-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g\" (UID: \"72bf5b60-00c5-4a80-9583-9dc1819afd99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:09.209153 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.209131    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72bf5b60-00c5-4a80-9583-9dc1819afd99-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g\" (UID: \"72bf5b60-00c5-4a80-9583-9dc1819afd99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:09.211840 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.211809    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72bf5b60-00c5-4a80-9583-9dc1819afd99-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g\" (UID: \"72bf5b60-00c5-4a80-9583-9dc1819afd99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:09.217553 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.217515    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6pc7\" (UniqueName: \"kubernetes.io/projected/72bf5b60-00c5-4a80-9583-9dc1819afd99-kube-api-access-f6pc7\") pod \"maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g\" (UID: \"72bf5b60-00c5-4a80-9583-9dc1819afd99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:09.370354 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.370279    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:09.546037 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.545943    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"]
Jul 27 15:58:09.549033 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:58:09.548981    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72bf5b60_00c5_4a80_9583_9dc1819afd99.slice/crio-d2ee28adc657b29b8334171e855bf8c621618b1e6c3b16614b6bda12072f890a WatchSource:0}: Error finding container d2ee28adc657b29b8334171e855bf8c621618b1e6c3b16614b6bda12072f890a: Status 404 returned error can't find the container with id d2ee28adc657b29b8334171e855bf8c621618b1e6c3b16614b6bda12072f890a
Jul 27 15:58:09.559500 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.559471    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:09.631636 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.631551    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:09.730351 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:09.730317    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:10.324205 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:10.324114    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g" event={"ID":"72bf5b60-00c5-4a80-9583-9dc1819afd99","Type":"ContainerStarted","Data":"0a4c45de557d1b433602eff63c327b44cc2b05054cb66cce8419ec05a3e8436f"}
Jul 27 15:58:10.324205 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:10.324176    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g" event={"ID":"72bf5b60-00c5-4a80-9583-9dc1819afd99","Type":"ContainerStarted","Data":"d2ee28adc657b29b8334171e855bf8c621618b1e6c3b16614b6bda12072f890a"}
Jul 27 15:58:10.324205 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:10.324204    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:10.352366 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:10.352314    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g" podStartSLOduration=0.884226136 podStartE2EDuration="1.352299588s" totalImagesPullingTime="468.073452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:58:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:58:10.351356421 +0000 UTC m=+3161.780031605" watchObservedRunningTime="2026-07-27 15:58:10.352299588 +0000 UTC m=+3161.780974770"
Jul 27 15:58:12.195548 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:12.195514    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:14.137610 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:14.137575    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kkdl4"]
Jul 27 15:58:14.141234 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:14.141213    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kkdl4"
Jul 27 15:58:14.151394 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:14.151365    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kkdl4"]
Jul 27 15:58:14.255686 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:14.255656    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv86l\" (UniqueName: \"kubernetes.io/projected/67f3d1fe-482c-4674-abfc-e3c04318a749-kube-api-access-kv86l\") pod \"kuadrant-operator-catalog-kkdl4\" (UID: \"67f3d1fe-482c-4674-abfc-e3c04318a749\") " pod="kuadrant-system/kuadrant-operator-catalog-kkdl4"
Jul 27 15:58:14.356556 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:14.356500    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv86l\" (UniqueName: \"kubernetes.io/projected/67f3d1fe-482c-4674-abfc-e3c04318a749-kube-api-access-kv86l\") pod \"kuadrant-operator-catalog-kkdl4\" (UID: \"67f3d1fe-482c-4674-abfc-e3c04318a749\") " pod="kuadrant-system/kuadrant-operator-catalog-kkdl4"
Jul 27 15:58:14.365445 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:14.365421    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv86l\" (UniqueName: \"kubernetes.io/projected/67f3d1fe-482c-4674-abfc-e3c04318a749-kube-api-access-kv86l\") pod \"kuadrant-operator-catalog-kkdl4\" (UID: \"67f3d1fe-482c-4674-abfc-e3c04318a749\") " pod="kuadrant-system/kuadrant-operator-catalog-kkdl4"
Jul 27 15:58:14.454001 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:14.453921    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kkdl4"
Jul 27 15:58:14.592279 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:14.592254    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kkdl4"]
Jul 27 15:58:14.594245 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:58:14.594214    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67f3d1fe_482c_4674_abfc_e3c04318a749.slice/crio-3bb0fe18c7f219040ffffdec6c3a31a79b98a2f53470d678e5f3dff3a9faeef6 WatchSource:0}: Error finding container 3bb0fe18c7f219040ffffdec6c3a31a79b98a2f53470d678e5f3dff3a9faeef6: Status 404 returned error can't find the container with id 3bb0fe18c7f219040ffffdec6c3a31a79b98a2f53470d678e5f3dff3a9faeef6
Jul 27 15:58:15.352273 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:15.352166    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kkdl4" event={"ID":"67f3d1fe-482c-4674-abfc-e3c04318a749","Type":"ContainerStarted","Data":"c1833209bbe1aa6cb0ccd016bee9ce7eeac5d42f8064136c8b3e6648f3df2449"}
Jul 27 15:58:15.352273 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:15.352222    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kkdl4" event={"ID":"67f3d1fe-482c-4674-abfc-e3c04318a749","Type":"ContainerStarted","Data":"3bb0fe18c7f219040ffffdec6c3a31a79b98a2f53470d678e5f3dff3a9faeef6"}
Jul 27 15:58:15.373316 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:15.373261    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-kkdl4" podStartSLOduration=0.975187701 podStartE2EDuration="1.373245552s" totalImagesPullingTime="398.057851ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:58:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:58:15.371082983 +0000 UTC m=+3166.799758167" watchObservedRunningTime="2026-07-27 15:58:15.373245552 +0000 UTC m=+3166.801920735"
Jul 27 15:58:16.337143 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:16.337115    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:23.139716 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:23.139678    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq"]
Jul 27 15:58:23.144060 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:23.144034    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq"
Jul 27 15:58:23.144816 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:23.144786    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq"]
Jul 27 15:58:23.146826 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:23.146796    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bgfwg\""
Jul 27 15:58:23.242815 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:23.242781    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3a9a15-0dcb-464b-a86c-276eda557610-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq\" (UID: \"ee3a9a15-0dcb-464b-a86c-276eda557610\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq"
Jul 27 15:58:23.343309 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:23.343274    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3a9a15-0dcb-464b-a86c-276eda557610-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq\" (UID: \"ee3a9a15-0dcb-464b-a86c-276eda557610\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq"
Jul 27 15:58:23.343945 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:23.343926    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3a9a15-0dcb-464b-a86c-276eda557610-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq\" (UID: \"ee3a9a15-0dcb-464b-a86c-276eda557610\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq"
Jul 27 15:58:23.456683 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:23.456589    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq"
Jul 27 15:58:23.597322 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:23.597289    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq"]
Jul 27 15:58:23.598107 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:58:23.598080    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee3a9a15_0dcb_464b_a86c_276eda557610.slice/crio-586e5ddfff6c6fec6d63cf37e8ba81d82f3572d4f0eb02c865eb72a36c3a1f67 WatchSource:0}: Error finding container 586e5ddfff6c6fec6d63cf37e8ba81d82f3572d4f0eb02c865eb72a36c3a1f67: Status 404 returned error can't find the container with id 586e5ddfff6c6fec6d63cf37e8ba81d82f3572d4f0eb02c865eb72a36c3a1f67
Jul 27 15:58:24.403235 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:24.403202    2548 generic.go:356] "Generic (PLEG): container finished" podID="ee3a9a15-0dcb-464b-a86c-276eda557610" containerID="50bb1f6531a5e9ad48a0f2da03ca7d64d8d7506c7ac758620d516badfefa354e" exitCode=0
Jul 27 15:58:24.403658 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:24.403288    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq" event={"ID":"ee3a9a15-0dcb-464b-a86c-276eda557610","Type":"ContainerDied","Data":"50bb1f6531a5e9ad48a0f2da03ca7d64d8d7506c7ac758620d516badfefa354e"}
Jul 27 15:58:24.403658 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:24.403334    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq" event={"ID":"ee3a9a15-0dcb-464b-a86c-276eda557610","Type":"ContainerStarted","Data":"586e5ddfff6c6fec6d63cf37e8ba81d82f3572d4f0eb02c865eb72a36c3a1f67"}
Jul 27 15:58:24.454395 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:24.454361    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-kkdl4"
Jul 27 15:58:24.454602 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:24.454424    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-kkdl4"
Jul 27 15:58:24.477162 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:24.477136    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-kkdl4"
Jul 27 15:58:25.441157 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:25.440817    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-kkdl4"
Jul 27 15:58:25.509835 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:25.509799    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kkdl4"]
Jul 27 15:58:25.554874 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:25.554846    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq"
Jul 27 15:58:25.671702 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:25.671622    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3a9a15-0dcb-464b-a86c-276eda557610-maas-api-service-ca\") pod \"ee3a9a15-0dcb-464b-a86c-276eda557610\" (UID: \"ee3a9a15-0dcb-464b-a86c-276eda557610\") "
Jul 27 15:58:25.671999 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:25.671974    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee3a9a15-0dcb-464b-a86c-276eda557610-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ee3a9a15-0dcb-464b-a86c-276eda557610" (UID: "ee3a9a15-0dcb-464b-a86c-276eda557610"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:58:25.772836 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:25.772797    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3a9a15-0dcb-464b-a86c-276eda557610-maas-api-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:58:26.416549 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:26.416510    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq"
Jul 27 15:58:26.416717 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:26.416555    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq" event={"ID":"ee3a9a15-0dcb-464b-a86c-276eda557610","Type":"ContainerDied","Data":"586e5ddfff6c6fec6d63cf37e8ba81d82f3572d4f0eb02c865eb72a36c3a1f67"}
Jul 27 15:58:26.416717 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:26.416610    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="586e5ddfff6c6fec6d63cf37e8ba81d82f3572d4f0eb02c865eb72a36c3a1f67"
Jul 27 15:58:27.421631 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:27.421589    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-kkdl4" podUID="67f3d1fe-482c-4674-abfc-e3c04318a749" containerName="registry-server" containerID="cri-o://c1833209bbe1aa6cb0ccd016bee9ce7eeac5d42f8064136c8b3e6648f3df2449" gracePeriod=2
Jul 27 15:58:27.679437 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:27.679377    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kkdl4"
Jul 27 15:58:27.792724 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:27.792685    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kv86l\" (UniqueName: \"kubernetes.io/projected/67f3d1fe-482c-4674-abfc-e3c04318a749-kube-api-access-kv86l\") pod \"67f3d1fe-482c-4674-abfc-e3c04318a749\" (UID: \"67f3d1fe-482c-4674-abfc-e3c04318a749\") "
Jul 27 15:58:27.795090 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:27.795049    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67f3d1fe-482c-4674-abfc-e3c04318a749-kube-api-access-kv86l" (OuterVolumeSpecName: "kube-api-access-kv86l") pod "67f3d1fe-482c-4674-abfc-e3c04318a749" (UID: "67f3d1fe-482c-4674-abfc-e3c04318a749"). InnerVolumeSpecName "kube-api-access-kv86l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:58:27.893992 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:27.893954    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv86l\" (UniqueName: \"kubernetes.io/projected/67f3d1fe-482c-4674-abfc-e3c04318a749-kube-api-access-kv86l\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:58:28.427793 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:28.427761    2548 generic.go:356] "Generic (PLEG): container finished" podID="67f3d1fe-482c-4674-abfc-e3c04318a749" containerID="c1833209bbe1aa6cb0ccd016bee9ce7eeac5d42f8064136c8b3e6648f3df2449" exitCode=0
Jul 27 15:58:28.428234 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:28.427801    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kkdl4"
Jul 27 15:58:28.428234 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:28.427855    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kkdl4" event={"ID":"67f3d1fe-482c-4674-abfc-e3c04318a749","Type":"ContainerDied","Data":"c1833209bbe1aa6cb0ccd016bee9ce7eeac5d42f8064136c8b3e6648f3df2449"}
Jul 27 15:58:28.428234 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:28.427917    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kkdl4" event={"ID":"67f3d1fe-482c-4674-abfc-e3c04318a749","Type":"ContainerDied","Data":"3bb0fe18c7f219040ffffdec6c3a31a79b98a2f53470d678e5f3dff3a9faeef6"}
Jul 27 15:58:28.428234 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:28.427949    2548 scope.go:117] "RemoveContainer" containerID="c1833209bbe1aa6cb0ccd016bee9ce7eeac5d42f8064136c8b3e6648f3df2449"
Jul 27 15:58:28.438669 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:28.438650    2548 scope.go:117] "RemoveContainer" containerID="c1833209bbe1aa6cb0ccd016bee9ce7eeac5d42f8064136c8b3e6648f3df2449"
Jul 27 15:58:28.438962 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:58:28.438943    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1833209bbe1aa6cb0ccd016bee9ce7eeac5d42f8064136c8b3e6648f3df2449\": container with ID starting with c1833209bbe1aa6cb0ccd016bee9ce7eeac5d42f8064136c8b3e6648f3df2449 not found: ID does not exist" containerID="c1833209bbe1aa6cb0ccd016bee9ce7eeac5d42f8064136c8b3e6648f3df2449"
Jul 27 15:58:28.439062 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:28.438970    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1833209bbe1aa6cb0ccd016bee9ce7eeac5d42f8064136c8b3e6648f3df2449"} err="failed to get container status \"c1833209bbe1aa6cb0ccd016bee9ce7eeac5d42f8064136c8b3e6648f3df2449\": rpc error: code = NotFound desc = could not find container \"c1833209bbe1aa6cb0ccd016bee9ce7eeac5d42f8064136c8b3e6648f3df2449\": container with ID starting with c1833209bbe1aa6cb0ccd016bee9ce7eeac5d42f8064136c8b3e6648f3df2449 not found: ID does not exist"
Jul 27 15:58:28.459557 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:28.459443    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kkdl4"]
Jul 27 15:58:28.462894 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:28.462871    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kkdl4"]
Jul 27 15:58:29.140724 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:29.140678    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67f3d1fe-482c-4674-abfc-e3c04318a749" path="/var/lib/kubelet/pods/67f3d1fe-482c-4674-abfc-e3c04318a749/volumes"
Jul 27 15:58:34.900905 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:34.900868    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"]
Jul 27 15:58:34.901400 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:34.901143    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g" podUID="72bf5b60-00c5-4a80-9583-9dc1819afd99" containerName="maas-api" containerID="cri-o://0a4c45de557d1b433602eff63c327b44cc2b05054cb66cce8419ec05a3e8436f" gracePeriod=30
Jul 27 15:58:34.935818 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:34.935790    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:35.115323 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.115292    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:35.160755 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.160689    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:35.182810 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.182778    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:35.260344 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.260307    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6pc7\" (UniqueName: \"kubernetes.io/projected/72bf5b60-00c5-4a80-9583-9dc1819afd99-kube-api-access-f6pc7\") pod \"72bf5b60-00c5-4a80-9583-9dc1819afd99\" (UID: \"72bf5b60-00c5-4a80-9583-9dc1819afd99\") "
Jul 27 15:58:35.260549 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.260426    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72bf5b60-00c5-4a80-9583-9dc1819afd99-openshift-service-ca\") pod \"72bf5b60-00c5-4a80-9583-9dc1819afd99\" (UID: \"72bf5b60-00c5-4a80-9583-9dc1819afd99\") "
Jul 27 15:58:35.260549 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.260477    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72bf5b60-00c5-4a80-9583-9dc1819afd99-maas-api-tls\") pod \"72bf5b60-00c5-4a80-9583-9dc1819afd99\" (UID: \"72bf5b60-00c5-4a80-9583-9dc1819afd99\") "
Jul 27 15:58:35.260822 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.260798    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72bf5b60-00c5-4a80-9583-9dc1819afd99-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "72bf5b60-00c5-4a80-9583-9dc1819afd99" (UID: "72bf5b60-00c5-4a80-9583-9dc1819afd99"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:58:35.260904 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.260880    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72bf5b60-00c5-4a80-9583-9dc1819afd99-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:58:35.262929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.262904    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72bf5b60-00c5-4a80-9583-9dc1819afd99-kube-api-access-f6pc7" (OuterVolumeSpecName: "kube-api-access-f6pc7") pod "72bf5b60-00c5-4a80-9583-9dc1819afd99" (UID: "72bf5b60-00c5-4a80-9583-9dc1819afd99"). InnerVolumeSpecName "kube-api-access-f6pc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:58:35.263036 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.262907    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72bf5b60-00c5-4a80-9583-9dc1819afd99-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "72bf5b60-00c5-4a80-9583-9dc1819afd99" (UID: "72bf5b60-00c5-4a80-9583-9dc1819afd99"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:58:35.361632 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.361590    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72bf5b60-00c5-4a80-9583-9dc1819afd99-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:58:35.361632 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.361627    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6pc7\" (UniqueName: \"kubernetes.io/projected/72bf5b60-00c5-4a80-9583-9dc1819afd99-kube-api-access-f6pc7\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:58:35.468280 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.468183    2548 generic.go:356] "Generic (PLEG): container finished" podID="72bf5b60-00c5-4a80-9583-9dc1819afd99" containerID="0a4c45de557d1b433602eff63c327b44cc2b05054cb66cce8419ec05a3e8436f" exitCode=0
Jul 27 15:58:35.468280 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.468236    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"
Jul 27 15:58:35.468523 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.468273    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g" event={"ID":"72bf5b60-00c5-4a80-9583-9dc1819afd99","Type":"ContainerDied","Data":"0a4c45de557d1b433602eff63c327b44cc2b05054cb66cce8419ec05a3e8436f"}
Jul 27 15:58:35.468523 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.468335    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g" event={"ID":"72bf5b60-00c5-4a80-9583-9dc1819afd99","Type":"ContainerDied","Data":"d2ee28adc657b29b8334171e855bf8c621618b1e6c3b16614b6bda12072f890a"}
Jul 27 15:58:35.468523 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.468365    2548 scope.go:117] "RemoveContainer" containerID="0a4c45de557d1b433602eff63c327b44cc2b05054cb66cce8419ec05a3e8436f"
Jul 27 15:58:35.478344 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.478325    2548 scope.go:117] "RemoveContainer" containerID="0a4c45de557d1b433602eff63c327b44cc2b05054cb66cce8419ec05a3e8436f"
Jul 27 15:58:35.478656 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:58:35.478637    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a4c45de557d1b433602eff63c327b44cc2b05054cb66cce8419ec05a3e8436f\": container with ID starting with 0a4c45de557d1b433602eff63c327b44cc2b05054cb66cce8419ec05a3e8436f not found: ID does not exist" containerID="0a4c45de557d1b433602eff63c327b44cc2b05054cb66cce8419ec05a3e8436f"
Jul 27 15:58:35.478716 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.478667    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a4c45de557d1b433602eff63c327b44cc2b05054cb66cce8419ec05a3e8436f"} err="failed to get container status \"0a4c45de557d1b433602eff63c327b44cc2b05054cb66cce8419ec05a3e8436f\": rpc error: code = NotFound desc = could not find container \"0a4c45de557d1b433602eff63c327b44cc2b05054cb66cce8419ec05a3e8436f\": container with ID starting with 0a4c45de557d1b433602eff63c327b44cc2b05054cb66cce8419ec05a3e8436f not found: ID does not exist"
Jul 27 15:58:35.492548 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.492501    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"]
Jul 27 15:58:35.496064 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:35.496041    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4cd197-5ddbb4b564-dk82g"]
Jul 27 15:58:37.139093 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:37.139055    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72bf5b60-00c5-4a80-9583-9dc1819afd99" path="/var/lib/kubelet/pods/72bf5b60-00c5-4a80-9583-9dc1819afd99/volumes"
Jul 27 15:58:38.230595 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:38.230563    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:38.247140 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:38.247113    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq"]
Jul 27 15:58:38.252959 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:38.252934    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4cd197-fjmpq"]
Jul 27 15:58:38.637487 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:38.637401    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:39.139751 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:39.139717    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee3a9a15-0dcb-464b-a86c-276eda557610" path="/var/lib/kubelet/pods/ee3a9a15-0dcb-464b-a86c-276eda557610/volumes"
Jul 27 15:58:45.315085 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:45.315055    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:45.415464 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:45.415427    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:45.604504 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:45.604419    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:45.831073 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:45.831032    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:49.318066 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.318037    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm"]
Jul 27 15:58:49.318461 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.318449    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee3a9a15-0dcb-464b-a86c-276eda557610" containerName="revoke-keys"
Jul 27 15:58:49.318514 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.318462    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee3a9a15-0dcb-464b-a86c-276eda557610" containerName="revoke-keys"
Jul 27 15:58:49.318514 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.318476    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72bf5b60-00c5-4a80-9583-9dc1819afd99" containerName="maas-api"
Jul 27 15:58:49.318514 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.318481    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="72bf5b60-00c5-4a80-9583-9dc1819afd99" containerName="maas-api"
Jul 27 15:58:49.318514 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.318491    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67f3d1fe-482c-4674-abfc-e3c04318a749" containerName="registry-server"
Jul 27 15:58:49.318514 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.318496    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="67f3d1fe-482c-4674-abfc-e3c04318a749" containerName="registry-server"
Jul 27 15:58:49.318717 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.318584    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="72bf5b60-00c5-4a80-9583-9dc1819afd99" containerName="maas-api"
Jul 27 15:58:49.318717 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.318594    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="67f3d1fe-482c-4674-abfc-e3c04318a749" containerName="registry-server"
Jul 27 15:58:49.318717 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.318601    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee3a9a15-0dcb-464b-a86c-276eda557610" containerName="revoke-keys"
Jul 27 15:58:49.321772 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.321756    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm"
Jul 27 15:58:49.324421 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.324402    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bgfwg\""
Jul 27 15:58:49.329024 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.328988    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm"]
Jul 27 15:58:49.500327 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.500283    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7c95664-fe06-423c-a0d7-3f3058b77261-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm\" (UID: \"f7c95664-fe06-423c-a0d7-3f3058b77261\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm"
Jul 27 15:58:49.601443 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.601353    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7c95664-fe06-423c-a0d7-3f3058b77261-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm\" (UID: \"f7c95664-fe06-423c-a0d7-3f3058b77261\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm"
Jul 27 15:58:49.601987 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.601962    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7c95664-fe06-423c-a0d7-3f3058b77261-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm\" (UID: \"f7c95664-fe06-423c-a0d7-3f3058b77261\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm"
Jul 27 15:58:49.634269 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.634241    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm"
Jul 27 15:58:49.800527 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:49.800495    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm"]
Jul 27 15:58:49.802364 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:58:49.802325    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7c95664_fe06_423c_a0d7_3f3058b77261.slice/crio-470d51d8e2f4a580043d27cd99234fd425ad27ed5277d093e149cb1d8fb9e19a WatchSource:0}: Error finding container 470d51d8e2f4a580043d27cd99234fd425ad27ed5277d093e149cb1d8fb9e19a: Status 404 returned error can't find the container with id 470d51d8e2f4a580043d27cd99234fd425ad27ed5277d093e149cb1d8fb9e19a
Jul 27 15:58:50.552767 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:50.552728    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm" event={"ID":"f7c95664-fe06-423c-a0d7-3f3058b77261","Type":"ContainerStarted","Data":"e1a98e4897ca458d8028d73ab8dddd12ffbfe9c45c6b23a514dee27328f44621"}
Jul 27 15:58:50.553175 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:50.552792    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm" event={"ID":"f7c95664-fe06-423c-a0d7-3f3058b77261","Type":"ContainerStarted","Data":"470d51d8e2f4a580043d27cd99234fd425ad27ed5277d093e149cb1d8fb9e19a"}
Jul 27 15:58:50.570469 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:50.570413    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm" podStartSLOduration=1.5703975350000001 podStartE2EDuration="1.570397535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:58:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:58:50.568333403 +0000 UTC m=+3201.997008589" watchObservedRunningTime="2026-07-27 15:58:50.570397535 +0000 UTC m=+3201.999072718"
Jul 27 15:58:51.559359 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:51.559326    2548 generic.go:356] "Generic (PLEG): container finished" podID="f7c95664-fe06-423c-a0d7-3f3058b77261" containerID="e1a98e4897ca458d8028d73ab8dddd12ffbfe9c45c6b23a514dee27328f44621" exitCode=0
Jul 27 15:58:51.559803 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:51.559383    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm" event={"ID":"f7c95664-fe06-423c-a0d7-3f3058b77261","Type":"ContainerDied","Data":"e1a98e4897ca458d8028d73ab8dddd12ffbfe9c45c6b23a514dee27328f44621"}
Jul 27 15:58:52.721036 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:52.721013    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm"
Jul 27 15:58:52.728934 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:52.728912    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7c95664-fe06-423c-a0d7-3f3058b77261-maas-api-service-ca\") pod \"f7c95664-fe06-423c-a0d7-3f3058b77261\" (UID: \"f7c95664-fe06-423c-a0d7-3f3058b77261\") "
Jul 27 15:58:52.729236 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:52.729215    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7c95664-fe06-423c-a0d7-3f3058b77261-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f7c95664-fe06-423c-a0d7-3f3058b77261" (UID: "f7c95664-fe06-423c-a0d7-3f3058b77261"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:58:52.830032 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:52.829945    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7c95664-fe06-423c-a0d7-3f3058b77261-maas-api-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:58:53.571750 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:53.571714    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm"
Jul 27 15:58:53.571750 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:53.571725    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm" event={"ID":"f7c95664-fe06-423c-a0d7-3f3058b77261","Type":"ContainerDied","Data":"470d51d8e2f4a580043d27cd99234fd425ad27ed5277d093e149cb1d8fb9e19a"}
Jul 27 15:58:53.572016 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:53.571807    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="470d51d8e2f4a580043d27cd99234fd425ad27ed5277d093e149cb1d8fb9e19a"
Jul 27 15:58:57.961529 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:57.961493    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:58:58.176364 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:58:58.176333    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:59:08.160555 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.160508    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"]
Jul 27 15:59:08.161044 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.160798    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q" podUID="d901721e-8301-441f-9a5c-cb9f627969ea" containerName="maas-api" containerID="cri-o://1351c1f50fc0e093b9f18a71e318647ec49031dea17f833a532a76e44cd8e7e1" gracePeriod=30
Jul 27 15:59:08.192814 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.192782    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:59:08.215160 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.215124    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"]
Jul 27 15:59:08.215405 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.215378    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk" podUID="31f31e42-c018-494e-b03b-6017768547c2" containerName="payload-processing" containerID="cri-o://376ff73aa8d021810060380a13b576a78eb4a07d71efbe95424d61a82b8cdb02" gracePeriod=30
Jul 27 15:59:08.226881 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.226832    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"]
Jul 27 15:59:08.227102 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.227076    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45" podUID="c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792" containerName="payload-pre-processing" containerID="cri-o://4174e229b161f1c96391d4f5a9dc8035b99629601e61b2b43cb7198c0ab6e9bb" gracePeriod=30
Jul 27 15:59:08.294671 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.294637    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:59:08.550984 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.550959    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"
Jul 27 15:59:08.556966 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.556946    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:59:08.559054 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.559039    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"
Jul 27 15:59:08.652711 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.652673    2548 generic.go:356] "Generic (PLEG): container finished" podID="c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792" containerID="4174e229b161f1c96391d4f5a9dc8035b99629601e61b2b43cb7198c0ab6e9bb" exitCode=0
Jul 27 15:59:08.652929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.652727    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"
Jul 27 15:59:08.652929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.652774    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45" event={"ID":"c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792","Type":"ContainerDied","Data":"4174e229b161f1c96391d4f5a9dc8035b99629601e61b2b43cb7198c0ab6e9bb"}
Jul 27 15:59:08.652929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.652829    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45" event={"ID":"c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792","Type":"ContainerDied","Data":"d2b87bf06f16b3fb404f53f833a6adc3b7d6e9f8579d478d2aefa3bfc6ff05ac"}
Jul 27 15:59:08.652929 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.652855    2548 scope.go:117] "RemoveContainer" containerID="4174e229b161f1c96391d4f5a9dc8035b99629601e61b2b43cb7198c0ab6e9bb"
Jul 27 15:59:08.654086 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.654059    2548 generic.go:356] "Generic (PLEG): container finished" podID="d901721e-8301-441f-9a5c-cb9f627969ea" containerID="1351c1f50fc0e093b9f18a71e318647ec49031dea17f833a532a76e44cd8e7e1" exitCode=0
Jul 27 15:59:08.654216 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.654093    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q" event={"ID":"d901721e-8301-441f-9a5c-cb9f627969ea","Type":"ContainerDied","Data":"1351c1f50fc0e093b9f18a71e318647ec49031dea17f833a532a76e44cd8e7e1"}
Jul 27 15:59:08.654216 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.654137    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q" event={"ID":"d901721e-8301-441f-9a5c-cb9f627969ea","Type":"ContainerDied","Data":"725a27be203d2f48f4b30e35673b1f3c091fb17e3bab6cea62773faed43c2195"}
Jul 27 15:59:08.654216 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.654100    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"
Jul 27 15:59:08.655436 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.655418    2548 generic.go:356] "Generic (PLEG): container finished" podID="31f31e42-c018-494e-b03b-6017768547c2" containerID="376ff73aa8d021810060380a13b576a78eb4a07d71efbe95424d61a82b8cdb02" exitCode=0
Jul 27 15:59:08.655525 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.655443    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk" event={"ID":"31f31e42-c018-494e-b03b-6017768547c2","Type":"ContainerDied","Data":"376ff73aa8d021810060380a13b576a78eb4a07d71efbe95424d61a82b8cdb02"}
Jul 27 15:59:08.655525 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.655463    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"
Jul 27 15:59:08.655525 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.655488    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk" event={"ID":"31f31e42-c018-494e-b03b-6017768547c2","Type":"ContainerDied","Data":"4ff42e3832bdbfd14ddbfcb2eba020d0457461e8d1b91e54b2203d7dcc1ea534"}
Jul 27 15:59:08.661783 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.661768    2548 scope.go:117] "RemoveContainer" containerID="4174e229b161f1c96391d4f5a9dc8035b99629601e61b2b43cb7198c0ab6e9bb"
Jul 27 15:59:08.662008 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:59:08.661986    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4174e229b161f1c96391d4f5a9dc8035b99629601e61b2b43cb7198c0ab6e9bb\": container with ID starting with 4174e229b161f1c96391d4f5a9dc8035b99629601e61b2b43cb7198c0ab6e9bb not found: ID does not exist" containerID="4174e229b161f1c96391d4f5a9dc8035b99629601e61b2b43cb7198c0ab6e9bb"
Jul 27 15:59:08.662068 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.662016    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4174e229b161f1c96391d4f5a9dc8035b99629601e61b2b43cb7198c0ab6e9bb"} err="failed to get container status \"4174e229b161f1c96391d4f5a9dc8035b99629601e61b2b43cb7198c0ab6e9bb\": rpc error: code = NotFound desc = could not find container \"4174e229b161f1c96391d4f5a9dc8035b99629601e61b2b43cb7198c0ab6e9bb\": container with ID starting with 4174e229b161f1c96391d4f5a9dc8035b99629601e61b2b43cb7198c0ab6e9bb not found: ID does not exist"
Jul 27 15:59:08.662068 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.662032    2548 scope.go:117] "RemoveContainer" containerID="1351c1f50fc0e093b9f18a71e318647ec49031dea17f833a532a76e44cd8e7e1"
Jul 27 15:59:08.670159 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.670141    2548 scope.go:117] "RemoveContainer" containerID="1351c1f50fc0e093b9f18a71e318647ec49031dea17f833a532a76e44cd8e7e1"
Jul 27 15:59:08.670397 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:59:08.670379    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1351c1f50fc0e093b9f18a71e318647ec49031dea17f833a532a76e44cd8e7e1\": container with ID starting with 1351c1f50fc0e093b9f18a71e318647ec49031dea17f833a532a76e44cd8e7e1 not found: ID does not exist" containerID="1351c1f50fc0e093b9f18a71e318647ec49031dea17f833a532a76e44cd8e7e1"
Jul 27 15:59:08.670460 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.670404    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1351c1f50fc0e093b9f18a71e318647ec49031dea17f833a532a76e44cd8e7e1"} err="failed to get container status \"1351c1f50fc0e093b9f18a71e318647ec49031dea17f833a532a76e44cd8e7e1\": rpc error: code = NotFound desc = could not find container \"1351c1f50fc0e093b9f18a71e318647ec49031dea17f833a532a76e44cd8e7e1\": container with ID starting with 1351c1f50fc0e093b9f18a71e318647ec49031dea17f833a532a76e44cd8e7e1 not found: ID does not exist"
Jul 27 15:59:08.670460 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.670426    2548 scope.go:117] "RemoveContainer" containerID="376ff73aa8d021810060380a13b576a78eb4a07d71efbe95424d61a82b8cdb02"
Jul 27 15:59:08.672660 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.672640    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rlr8\" (UniqueName: \"kubernetes.io/projected/c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792-kube-api-access-5rlr8\") pod \"c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792\" (UID: \"c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792\") "
Jul 27 15:59:08.672776 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.672682    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792-plugins-config-volume\") pod \"c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792\" (UID: \"c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792\") "
Jul 27 15:59:08.672776 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.672724    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d901721e-8301-441f-9a5c-cb9f627969ea-maas-api-tls\") pod \"d901721e-8301-441f-9a5c-cb9f627969ea\" (UID: \"d901721e-8301-441f-9a5c-cb9f627969ea\") "
Jul 27 15:59:08.672776 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.672756    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31f31e42-c018-494e-b03b-6017768547c2-plugins-config-volume\") pod \"31f31e42-c018-494e-b03b-6017768547c2\" (UID: \"31f31e42-c018-494e-b03b-6017768547c2\") "
Jul 27 15:59:08.672934 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.672812    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wk7wc\" (UniqueName: \"kubernetes.io/projected/d901721e-8301-441f-9a5c-cb9f627969ea-kube-api-access-wk7wc\") pod \"d901721e-8301-441f-9a5c-cb9f627969ea\" (UID: \"d901721e-8301-441f-9a5c-cb9f627969ea\") "
Jul 27 15:59:08.672934 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.672841    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxjdz\" (UniqueName: \"kubernetes.io/projected/31f31e42-c018-494e-b03b-6017768547c2-kube-api-access-bxjdz\") pod \"31f31e42-c018-494e-b03b-6017768547c2\" (UID: \"31f31e42-c018-494e-b03b-6017768547c2\") "
Jul 27 15:59:08.672934 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.672920    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d901721e-8301-441f-9a5c-cb9f627969ea-openshift-service-ca\") pod \"d901721e-8301-441f-9a5c-cb9f627969ea\" (UID: \"d901721e-8301-441f-9a5c-cb9f627969ea\") "
Jul 27 15:59:08.673160 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.673135    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792" (UID: "c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:59:08.673310 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.673289    2548 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792-plugins-config-volume\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:59:08.673378 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.673167    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31f31e42-c018-494e-b03b-6017768547c2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "31f31e42-c018-494e-b03b-6017768547c2" (UID: "31f31e42-c018-494e-b03b-6017768547c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:59:08.673634 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.673610    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d901721e-8301-441f-9a5c-cb9f627969ea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d901721e-8301-441f-9a5c-cb9f627969ea" (UID: "d901721e-8301-441f-9a5c-cb9f627969ea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 15:59:08.675070 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.675051    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d901721e-8301-441f-9a5c-cb9f627969ea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d901721e-8301-441f-9a5c-cb9f627969ea" (UID: "d901721e-8301-441f-9a5c-cb9f627969ea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 15:59:08.675167 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.675080    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792-kube-api-access-5rlr8" (OuterVolumeSpecName: "kube-api-access-5rlr8") pod "c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792" (UID: "c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792"). InnerVolumeSpecName "kube-api-access-5rlr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:59:08.675383 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.675367    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d901721e-8301-441f-9a5c-cb9f627969ea-kube-api-access-wk7wc" (OuterVolumeSpecName: "kube-api-access-wk7wc") pod "d901721e-8301-441f-9a5c-cb9f627969ea" (UID: "d901721e-8301-441f-9a5c-cb9f627969ea"). InnerVolumeSpecName "kube-api-access-wk7wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:59:08.675761 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.675738    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31f31e42-c018-494e-b03b-6017768547c2-kube-api-access-bxjdz" (OuterVolumeSpecName: "kube-api-access-bxjdz") pod "31f31e42-c018-494e-b03b-6017768547c2" (UID: "31f31e42-c018-494e-b03b-6017768547c2"). InnerVolumeSpecName "kube-api-access-bxjdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 15:59:08.679979 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.679956    2548 scope.go:117] "RemoveContainer" containerID="376ff73aa8d021810060380a13b576a78eb4a07d71efbe95424d61a82b8cdb02"
Jul 27 15:59:08.680238 ip-10-0-143-46 kubenswrapper[2548]: E0727 15:59:08.680221    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"376ff73aa8d021810060380a13b576a78eb4a07d71efbe95424d61a82b8cdb02\": container with ID starting with 376ff73aa8d021810060380a13b576a78eb4a07d71efbe95424d61a82b8cdb02 not found: ID does not exist" containerID="376ff73aa8d021810060380a13b576a78eb4a07d71efbe95424d61a82b8cdb02"
Jul 27 15:59:08.680291 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.680247    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"376ff73aa8d021810060380a13b576a78eb4a07d71efbe95424d61a82b8cdb02"} err="failed to get container status \"376ff73aa8d021810060380a13b576a78eb4a07d71efbe95424d61a82b8cdb02\": rpc error: code = NotFound desc = could not find container \"376ff73aa8d021810060380a13b576a78eb4a07d71efbe95424d61a82b8cdb02\": container with ID starting with 376ff73aa8d021810060380a13b576a78eb4a07d71efbe95424d61a82b8cdb02 not found: ID does not exist"
Jul 27 15:59:08.774807 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.774773    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d901721e-8301-441f-9a5c-cb9f627969ea-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:59:08.774807 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.774800    2548 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31f31e42-c018-494e-b03b-6017768547c2-plugins-config-volume\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:59:08.774807 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.774809    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wk7wc\" (UniqueName: \"kubernetes.io/projected/d901721e-8301-441f-9a5c-cb9f627969ea-kube-api-access-wk7wc\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:59:08.774807 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.774818    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxjdz\" (UniqueName: \"kubernetes.io/projected/31f31e42-c018-494e-b03b-6017768547c2-kube-api-access-bxjdz\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:59:08.775078 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.774826    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d901721e-8301-441f-9a5c-cb9f627969ea-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:59:08.775078 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.774835    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rlr8\" (UniqueName: \"kubernetes.io/projected/c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792-kube-api-access-5rlr8\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 15:59:08.982744 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.982708    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"]
Jul 27 15:59:08.988210 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:08.988183    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c4dbeb-5f8f58cc97-cmv8q"]
Jul 27 15:59:09.000494 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:09.000463    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"]
Jul 27 15:59:09.005967 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:09.005941    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c4dbeb-5ccb489cf6-scm45"]
Jul 27 15:59:09.017238 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:09.017213    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"]
Jul 27 15:59:09.024123 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:09.024096    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c4dbeb-75f5bcb6b-kttnk"]
Jul 27 15:59:09.138603 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:09.138568    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31f31e42-c018-494e-b03b-6017768547c2" path="/var/lib/kubelet/pods/31f31e42-c018-494e-b03b-6017768547c2/volumes"
Jul 27 15:59:09.138957 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:09.138944    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792" path="/var/lib/kubelet/pods/c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792/volumes"
Jul 27 15:59:09.139302 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:09.139287    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d901721e-8301-441f-9a5c-cb9f627969ea" path="/var/lib/kubelet/pods/d901721e-8301-441f-9a5c-cb9f627969ea/volumes"
Jul 27 15:59:09.393304 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:09.393243    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:59:09.394242 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:09.394215    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm"]
Jul 27 15:59:09.396032 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:09.396008    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c4dbeb-774pm"]
Jul 27 15:59:10.554848 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:10.554817    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:59:11.139634 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:11.139604    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7c95664-fe06-423c-a0d7-3f3058b77261" path="/var/lib/kubelet/pods/f7c95664-fe06-423c-a0d7-3f3058b77261/volumes"
Jul 27 15:59:14.812985 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:14.812954    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:59:39.627297 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:39.627263    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:59:50.903720 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:50.903684    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:59:52.361245 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:52.361207    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:59:58.384050 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:58.384019    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:59:58.481336 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:58.481308    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:59:58.657667 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:58.657572    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:59:58.909837 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:58.909749    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 15:59:59.030050 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.030015    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd"]
Jul 27 15:59:59.030460 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.030443    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792" containerName="payload-pre-processing"
Jul 27 15:59:59.030514 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.030462    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792" containerName="payload-pre-processing"
Jul 27 15:59:59.030514 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.030478    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c95664-fe06-423c-a0d7-3f3058b77261" containerName="revoke-keys"
Jul 27 15:59:59.030514 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.030483    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c95664-fe06-423c-a0d7-3f3058b77261" containerName="revoke-keys"
Jul 27 15:59:59.030514 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.030492    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31f31e42-c018-494e-b03b-6017768547c2" containerName="payload-processing"
Jul 27 15:59:59.030514 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.030497    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="31f31e42-c018-494e-b03b-6017768547c2" containerName="payload-processing"
Jul 27 15:59:59.030514 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.030504    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d901721e-8301-441f-9a5c-cb9f627969ea" containerName="maas-api"
Jul 27 15:59:59.030514 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.030509    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d901721e-8301-441f-9a5c-cb9f627969ea" containerName="maas-api"
Jul 27 15:59:59.030793 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.030585    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6482f1c-3c14-4dcf-bfdd-b3a3a45f0792" containerName="payload-pre-processing"
Jul 27 15:59:59.030793 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.030596    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7c95664-fe06-423c-a0d7-3f3058b77261" containerName="revoke-keys"
Jul 27 15:59:59.030793 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.030602    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="d901721e-8301-441f-9a5c-cb9f627969ea" containerName="maas-api"
Jul 27 15:59:59.030793 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.030607    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="31f31e42-c018-494e-b03b-6017768547c2" containerName="payload-processing"
Jul 27 15:59:59.033508 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.033490    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd"
Jul 27 15:59:59.037060 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.037041    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bgfwg\""
Jul 27 15:59:59.043905 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.043883    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd"]
Jul 27 15:59:59.134212 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.134178    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70afda37-1e58-4caa-be32-7e7e7ea3d5ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd\" (UID: \"70afda37-1e58-4caa-be32-7e7e7ea3d5ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd"
Jul 27 15:59:59.235545 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.235507    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70afda37-1e58-4caa-be32-7e7e7ea3d5ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd\" (UID: \"70afda37-1e58-4caa-be32-7e7e7ea3d5ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd"
Jul 27 15:59:59.236132 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.236111    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70afda37-1e58-4caa-be32-7e7e7ea3d5ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd\" (UID: \"70afda37-1e58-4caa-be32-7e7e7ea3d5ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd"
Jul 27 15:59:59.344627 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.344599    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd"
Jul 27 15:59:59.486715 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.486686    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd"]
Jul 27 15:59:59.488831 ip-10-0-143-46 kubenswrapper[2548]: W0727 15:59:59.488804    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70afda37_1e58_4caa_be32_7e7e7ea3d5ba.slice/crio-540172cb059cc6638de0bd03b096a58e55009a9774396ab3de5af9ab433e4912 WatchSource:0}: Error finding container 540172cb059cc6638de0bd03b096a58e55009a9774396ab3de5af9ab433e4912: Status 404 returned error can't find the container with id 540172cb059cc6638de0bd03b096a58e55009a9774396ab3de5af9ab433e4912
Jul 27 15:59:59.926369 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.926334    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd" event={"ID":"70afda37-1e58-4caa-be32-7e7e7ea3d5ba","Type":"ContainerStarted","Data":"a032160b554d4558da320dcd24bef56dd8e8596cd75132af8153d227446485f1"}
Jul 27 15:59:59.926575 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.926400    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd" event={"ID":"70afda37-1e58-4caa-be32-7e7e7ea3d5ba","Type":"ContainerStarted","Data":"540172cb059cc6638de0bd03b096a58e55009a9774396ab3de5af9ab433e4912"}
Jul 27 15:59:59.952956 ip-10-0-143-46 kubenswrapper[2548]: I0727 15:59:59.952905    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd" podStartSLOduration=0.952890357 podStartE2EDuration="952.890357ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 15:59:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 15:59:59.949619507 +0000 UTC m=+3271.378294690" watchObservedRunningTime="2026-07-27 15:59:59.952890357 +0000 UTC m=+3271.381565534"
Jul 27 16:00:00.141528 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.141384    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-052d16-29752800-9gjwn"]
Jul 27 16:00:00.145501 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.145457    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f58a29-29752800-ghkcq"]
Jul 27 16:00:00.146258 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.146234    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-052d16-29752800-9gjwn"
Jul 27 16:00:00.150968 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.150943    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752800-w8dph"]
Jul 27 16:00:00.151657 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.151634    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f58a29-29752800-ghkcq"
Jul 27 16:00:00.157251 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.157225    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-809b00-29752800-p8crm"]
Jul 27 16:00:00.157611 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.157591    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752800-w8dph"
Jul 27 16:00:00.161586 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.161139    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-052d16-29752800-9gjwn"]
Jul 27 16:00:00.161586 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.161218    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f58a29-29752800-ghkcq"]
Jul 27 16:00:00.161586 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.161305    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-809b00-29752800-p8crm"
Jul 27 16:00:00.162943 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.162069    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752800-w8dph"]
Jul 27 16:00:00.171369 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.171350    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-052d16-29752800-9gjwn"
Jul 27 16:00:00.175295 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.175276    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f58a29-29752800-ghkcq"
Jul 27 16:00:00.176928 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.176905    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-809b00-29752800-p8crm"]
Jul 27 16:00:00.185915 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.185893    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752800-w8dph"
Jul 27 16:00:00.210572 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.210550    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-809b00-29752800-p8crm"
Jul 27 16:00:00.367830 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.367803    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f58a29-29752800-ghkcq"]
Jul 27 16:00:00.369353 ip-10-0-143-46 kubenswrapper[2548]: W0727 16:00:00.369308    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca6ff2bd_f602_43bb_98b4_ba8236256f05.slice/crio-78a569e6a02c16a791b2b4636fe2c1fd9e26774455b8cf166b56d4dc12e76095 WatchSource:0}: Error finding container 78a569e6a02c16a791b2b4636fe2c1fd9e26774455b8cf166b56d4dc12e76095: Status 404 returned error can't find the container with id 78a569e6a02c16a791b2b4636fe2c1fd9e26774455b8cf166b56d4dc12e76095
Jul 27 16:00:00.432436 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.432410    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-809b00-29752800-p8crm"]
Jul 27 16:00:00.434020 ip-10-0-143-46 kubenswrapper[2548]: W0727 16:00:00.433997    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec83830a_2c36_4b82_b246_c81302d15d53.slice/crio-b814ef276487f9b249560f7cf533be07fe95d622915ed2a453cc8987cf8daa62 WatchSource:0}: Error finding container b814ef276487f9b249560f7cf533be07fe95d622915ed2a453cc8987cf8daa62: Status 404 returned error can't find the container with id b814ef276487f9b249560f7cf533be07fe95d622915ed2a453cc8987cf8daa62
Jul 27 16:00:00.594090 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.594066    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-052d16-29752800-9gjwn"]
Jul 27 16:00:00.597825 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.597803    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752800-w8dph"]
Jul 27 16:00:00.603844 ip-10-0-143-46 kubenswrapper[2548]: W0727 16:00:00.603817    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b364c2_3223_4be3_af53_a31b213944a7.slice/crio-2657dc61cc1e7994ec26f521b06f7d71de8bb6ccd0fc50ff581e9e4495502f5c WatchSource:0}: Error finding container 2657dc61cc1e7994ec26f521b06f7d71de8bb6ccd0fc50ff581e9e4495502f5c: Status 404 returned error can't find the container with id 2657dc61cc1e7994ec26f521b06f7d71de8bb6ccd0fc50ff581e9e4495502f5c
Jul 27 16:00:00.604114 ip-10-0-143-46 kubenswrapper[2548]: W0727 16:00:00.604085    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97fa695b_de5e_4165_bf01_374ac513e2e1.slice/crio-f8df46b1462cca0f56b8d5f6a04343a4cbc06e42781eab47aaeb2464bff98e90 WatchSource:0}: Error finding container f8df46b1462cca0f56b8d5f6a04343a4cbc06e42781eab47aaeb2464bff98e90: Status 404 returned error can't find the container with id f8df46b1462cca0f56b8d5f6a04343a4cbc06e42781eab47aaeb2464bff98e90
Jul 27 16:00:00.933266 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.933240    2548 generic.go:356] "Generic (PLEG): container finished" podID="70afda37-1e58-4caa-be32-7e7e7ea3d5ba" containerID="a032160b554d4558da320dcd24bef56dd8e8596cd75132af8153d227446485f1" exitCode=0
Jul 27 16:00:00.933454 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.933304    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd" event={"ID":"70afda37-1e58-4caa-be32-7e7e7ea3d5ba","Type":"ContainerDied","Data":"a032160b554d4558da320dcd24bef56dd8e8596cd75132af8153d227446485f1"}
Jul 27 16:00:00.934588 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.934564    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752800-w8dph" event={"ID":"97fa695b-de5e-4165-bf01-374ac513e2e1","Type":"ContainerStarted","Data":"f8df46b1462cca0f56b8d5f6a04343a4cbc06e42781eab47aaeb2464bff98e90"}
Jul 27 16:00:00.936195 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.936171    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-809b00-29752800-p8crm" event={"ID":"ec83830a-2c36-4b82-b246-c81302d15d53","Type":"ContainerStarted","Data":"339b4428ed6231590dfbe71130944d48e0cce2a8849927f6a065490fbd09c313"}
Jul 27 16:00:00.936294 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.936217    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-809b00-29752800-p8crm" event={"ID":"ec83830a-2c36-4b82-b246-c81302d15d53","Type":"ContainerStarted","Data":"b814ef276487f9b249560f7cf533be07fe95d622915ed2a453cc8987cf8daa62"}
Jul 27 16:00:00.937835 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.937814    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f58a29-29752800-ghkcq" event={"ID":"ca6ff2bd-f602-43bb-98b4-ba8236256f05","Type":"ContainerStarted","Data":"09324b6838c527678605e08acc55d41c112e473a54c738c0458464fe8bd3e3d3"}
Jul 27 16:00:00.937934 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.937849    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f58a29-29752800-ghkcq" event={"ID":"ca6ff2bd-f602-43bb-98b4-ba8236256f05","Type":"ContainerStarted","Data":"78a569e6a02c16a791b2b4636fe2c1fd9e26774455b8cf166b56d4dc12e76095"}
Jul 27 16:00:00.939275 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.939257    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-052d16-29752800-9gjwn" event={"ID":"01b364c2-3223-4be3-af53-a31b213944a7","Type":"ContainerStarted","Data":"fbea28a0589613a70de852e3f84a5a61593235fcf485a8b744685869f6f3410e"}
Jul 27 16:00:00.939362 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.939291    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-052d16-29752800-9gjwn" event={"ID":"01b364c2-3223-4be3-af53-a31b213944a7","Type":"ContainerStarted","Data":"2657dc61cc1e7994ec26f521b06f7d71de8bb6ccd0fc50ff581e9e4495502f5c"}
Jul 27 16:00:00.973000 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.972955    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-052d16-29752800-9gjwn" podStartSLOduration=0.972942761 podStartE2EDuration="972.942761ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:00:00.970361992 +0000 UTC m=+3272.399037175" watchObservedRunningTime="2026-07-27 16:00:00.972942761 +0000 UTC m=+3272.401617941"
Jul 27 16:00:00.990932 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:00.990890    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f58a29-29752800-ghkcq" podStartSLOduration=0.990876393 podStartE2EDuration="990.876393ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:00:00.98980359 +0000 UTC m=+3272.418478774" watchObservedRunningTime="2026-07-27 16:00:00.990876393 +0000 UTC m=+3272.419551611"
Jul 27 16:00:01.005972 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:01.005926    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-809b00-29752800-p8crm" podStartSLOduration=1.005914209 podStartE2EDuration="1.005914209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:00:01.004641597 +0000 UTC m=+3272.433316780" watchObservedRunningTime="2026-07-27 16:00:01.005914209 +0000 UTC m=+3272.434589391"
Jul 27 16:00:01.949647 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:01.949612    2548 generic.go:356] "Generic (PLEG): container finished" podID="97fa695b-de5e-4165-bf01-374ac513e2e1" containerID="99c5cb4e4b3655a0578c2cc611d5e686250b108f04722ee83dd856cdc55383a8" exitCode=0
Jul 27 16:00:01.950044 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:01.949705    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752800-w8dph" event={"ID":"97fa695b-de5e-4165-bf01-374ac513e2e1","Type":"ContainerDied","Data":"99c5cb4e4b3655a0578c2cc611d5e686250b108f04722ee83dd856cdc55383a8"}
Jul 27 16:00:01.951185 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:01.951155    2548 generic.go:356] "Generic (PLEG): container finished" podID="ec83830a-2c36-4b82-b246-c81302d15d53" containerID="339b4428ed6231590dfbe71130944d48e0cce2a8849927f6a065490fbd09c313" exitCode=0
Jul 27 16:00:01.951307 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:01.951233    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-809b00-29752800-p8crm" event={"ID":"ec83830a-2c36-4b82-b246-c81302d15d53","Type":"ContainerDied","Data":"339b4428ed6231590dfbe71130944d48e0cce2a8849927f6a065490fbd09c313"}
Jul 27 16:00:01.952686 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:01.952664    2548 generic.go:356] "Generic (PLEG): container finished" podID="ca6ff2bd-f602-43bb-98b4-ba8236256f05" containerID="09324b6838c527678605e08acc55d41c112e473a54c738c0458464fe8bd3e3d3" exitCode=0
Jul 27 16:00:01.952802 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:01.952748    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f58a29-29752800-ghkcq" event={"ID":"ca6ff2bd-f602-43bb-98b4-ba8236256f05","Type":"ContainerDied","Data":"09324b6838c527678605e08acc55d41c112e473a54c738c0458464fe8bd3e3d3"}
Jul 27 16:00:01.954323 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:01.954302    2548 generic.go:356] "Generic (PLEG): container finished" podID="01b364c2-3223-4be3-af53-a31b213944a7" containerID="fbea28a0589613a70de852e3f84a5a61593235fcf485a8b744685869f6f3410e" exitCode=0
Jul 27 16:00:01.954413 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:01.954360    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-052d16-29752800-9gjwn" event={"ID":"01b364c2-3223-4be3-af53-a31b213944a7","Type":"ContainerDied","Data":"fbea28a0589613a70de852e3f84a5a61593235fcf485a8b744685869f6f3410e"}
Jul 27 16:00:02.091644 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:02.091612    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd"
Jul 27 16:00:02.166524 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:02.166445    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70afda37-1e58-4caa-be32-7e7e7ea3d5ba-maas-api-service-ca\") pod \"70afda37-1e58-4caa-be32-7e7e7ea3d5ba\" (UID: \"70afda37-1e58-4caa-be32-7e7e7ea3d5ba\") "
Jul 27 16:00:02.166851 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:02.166826    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70afda37-1e58-4caa-be32-7e7e7ea3d5ba-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "70afda37-1e58-4caa-be32-7e7e7ea3d5ba" (UID: "70afda37-1e58-4caa-be32-7e7e7ea3d5ba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 16:00:02.268112 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:02.268071    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70afda37-1e58-4caa-be32-7e7e7ea3d5ba-maas-api-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 16:00:02.960918 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:02.960887    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd"
Jul 27 16:00:02.961384 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:02.960944    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd" event={"ID":"70afda37-1e58-4caa-be32-7e7e7ea3d5ba","Type":"ContainerDied","Data":"540172cb059cc6638de0bd03b096a58e55009a9774396ab3de5af9ab433e4912"}
Jul 27 16:00:02.961384 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:02.960983    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="540172cb059cc6638de0bd03b096a58e55009a9774396ab3de5af9ab433e4912"
Jul 27 16:00:03.125994 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.125965    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-809b00-29752800-p8crm"
Jul 27 16:00:03.188860 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.188830    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752800-w8dph"
Jul 27 16:00:03.192135 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.192112    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f58a29-29752800-ghkcq"
Jul 27 16:00:03.200667 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.200647    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-052d16-29752800-9gjwn"
Jul 27 16:00:03.969272 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.969245    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752800-w8dph"
Jul 27 16:00:03.969808 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.969275    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752800-w8dph" event={"ID":"97fa695b-de5e-4165-bf01-374ac513e2e1","Type":"ContainerDied","Data":"f8df46b1462cca0f56b8d5f6a04343a4cbc06e42781eab47aaeb2464bff98e90"}
Jul 27 16:00:03.969808 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.969358    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f8df46b1462cca0f56b8d5f6a04343a4cbc06e42781eab47aaeb2464bff98e90"
Jul 27 16:00:03.970985 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.970963    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-809b00-29752800-p8crm"
Jul 27 16:00:03.970985 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.970972    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-809b00-29752800-p8crm" event={"ID":"ec83830a-2c36-4b82-b246-c81302d15d53","Type":"ContainerDied","Data":"b814ef276487f9b249560f7cf533be07fe95d622915ed2a453cc8987cf8daa62"}
Jul 27 16:00:03.971172 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.971012    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b814ef276487f9b249560f7cf533be07fe95d622915ed2a453cc8987cf8daa62"
Jul 27 16:00:03.972489 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.972462    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f58a29-29752800-ghkcq"
Jul 27 16:00:03.972623 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.972515    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f58a29-29752800-ghkcq" event={"ID":"ca6ff2bd-f602-43bb-98b4-ba8236256f05","Type":"ContainerDied","Data":"78a569e6a02c16a791b2b4636fe2c1fd9e26774455b8cf166b56d4dc12e76095"}
Jul 27 16:00:03.972623 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.972581    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78a569e6a02c16a791b2b4636fe2c1fd9e26774455b8cf166b56d4dc12e76095"
Jul 27 16:00:03.974485 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.974463    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-052d16-29752800-9gjwn" event={"ID":"01b364c2-3223-4be3-af53-a31b213944a7","Type":"ContainerDied","Data":"2657dc61cc1e7994ec26f521b06f7d71de8bb6ccd0fc50ff581e9e4495502f5c"}
Jul 27 16:00:03.974599 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.974502    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2657dc61cc1e7994ec26f521b06f7d71de8bb6ccd0fc50ff581e9e4495502f5c"
Jul 27 16:00:03.974599 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:03.974579    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-052d16-29752800-9gjwn"
Jul 27 16:00:10.551421 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:10.551381    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"]
Jul 27 16:00:10.552560 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:10.552503    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk" podUID="bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442" containerName="maas-api" containerID="cri-o://f492fbbfdc974a9d23dde59a18a1709641836732a201cbefea6a00380687a37f" gracePeriod=30
Jul 27 16:00:10.579625 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:10.579594    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-809b00-29752800-p8crm"]
Jul 27 16:00:10.582500 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:10.582476    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-809b00-29752800-p8crm"]
Jul 27 16:00:10.812884 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:10.812827    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 16:00:10.944942 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:10.944908    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjqfj\" (UniqueName: \"kubernetes.io/projected/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-kube-api-access-qjqfj\") pod \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\" (UID: \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\") "
Jul 27 16:00:10.945118 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:10.944973    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-maas-api-tls\") pod \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\" (UID: \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\") "
Jul 27 16:00:10.945118 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:10.945052    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-openshift-service-ca\") pod \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\" (UID: \"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442\") "
Jul 27 16:00:10.945415 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:10.945391    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442" (UID: "bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 16:00:10.947256 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:10.947230    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-kube-api-access-qjqfj" (OuterVolumeSpecName: "kube-api-access-qjqfj") pod "bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442" (UID: "bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442"). InnerVolumeSpecName "kube-api-access-qjqfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 16:00:10.947352 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:10.947306    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442" (UID: "bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 16:00:11.011933 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.011895    2548 generic.go:356] "Generic (PLEG): container finished" podID="bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442" containerID="f492fbbfdc974a9d23dde59a18a1709641836732a201cbefea6a00380687a37f" exitCode=0
Jul 27 16:00:11.011933 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.011938    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"
Jul 27 16:00:11.012176 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.011979    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk" event={"ID":"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442","Type":"ContainerDied","Data":"f492fbbfdc974a9d23dde59a18a1709641836732a201cbefea6a00380687a37f"}
Jul 27 16:00:11.012176 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.012032    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk" event={"ID":"bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442","Type":"ContainerDied","Data":"6e222c7a4795bd98a9ee27b7012eed81434b492351c2f37c925dc315c7f21c1c"}
Jul 27 16:00:11.012176 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.012055    2548 scope.go:117] "RemoveContainer" containerID="f492fbbfdc974a9d23dde59a18a1709641836732a201cbefea6a00380687a37f"
Jul 27 16:00:11.021653 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.021590    2548 scope.go:117] "RemoveContainer" containerID="f492fbbfdc974a9d23dde59a18a1709641836732a201cbefea6a00380687a37f"
Jul 27 16:00:11.021900 ip-10-0-143-46 kubenswrapper[2548]: E0727 16:00:11.021880    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f492fbbfdc974a9d23dde59a18a1709641836732a201cbefea6a00380687a37f\": container with ID starting with f492fbbfdc974a9d23dde59a18a1709641836732a201cbefea6a00380687a37f not found: ID does not exist" containerID="f492fbbfdc974a9d23dde59a18a1709641836732a201cbefea6a00380687a37f"
Jul 27 16:00:11.021964 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.021910    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f492fbbfdc974a9d23dde59a18a1709641836732a201cbefea6a00380687a37f"} err="failed to get container status \"f492fbbfdc974a9d23dde59a18a1709641836732a201cbefea6a00380687a37f\": rpc error: code = NotFound desc = could not find container \"f492fbbfdc974a9d23dde59a18a1709641836732a201cbefea6a00380687a37f\": container with ID starting with f492fbbfdc974a9d23dde59a18a1709641836732a201cbefea6a00380687a37f not found: ID does not exist"
Jul 27 16:00:11.034939 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.034914    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"]
Jul 27 16:00:11.038661 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.038637    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-809b00-75f5d4d9bc-tkvlk"]
Jul 27 16:00:11.046019 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.046001    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjqfj\" (UniqueName: \"kubernetes.io/projected/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-kube-api-access-qjqfj\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 16:00:11.046081 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.046024    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 16:00:11.046081 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.046039    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 16:00:11.081959 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.081900    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:11.139846 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.139817    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442" path="/var/lib/kubelet/pods/bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442/volumes"
Jul 27 16:00:11.140107 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.140095    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec83830a-2c36-4b82-b246-c81302d15d53" path="/var/lib/kubelet/pods/ec83830a-2c36-4b82-b246-c81302d15d53/volumes"
Jul 27 16:00:11.183088 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:11.183063    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:14.089485 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:14.089446    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd"]
Jul 27 16:00:14.094710 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:14.094683    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-809b00-r8jdd"]
Jul 27 16:00:14.097083 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:14.097056    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:14.473857 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:14.473780    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:15.139883 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:15.139848    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70afda37-1e58-4caa-be32-7e7e7ea3d5ba" path="/var/lib/kubelet/pods/70afda37-1e58-4caa-be32-7e7e7ea3d5ba/volumes"
Jul 27 16:00:20.806439 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:20.806396    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:20.902821 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:20.902787    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:21.065641 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.065561    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:21.429767 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.429689    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw"]
Jul 27 16:00:21.430091 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430078    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01b364c2-3223-4be3-af53-a31b213944a7" containerName="cleanup"
Jul 27 16:00:21.430140 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430092    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="01b364c2-3223-4be3-af53-a31b213944a7" containerName="cleanup"
Jul 27 16:00:21.430140 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430109    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442" containerName="maas-api"
Jul 27 16:00:21.430140 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430115    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442" containerName="maas-api"
Jul 27 16:00:21.430140 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430130    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec83830a-2c36-4b82-b246-c81302d15d53" containerName="cleanup"
Jul 27 16:00:21.430140 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430135    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec83830a-2c36-4b82-b246-c81302d15d53" containerName="cleanup"
Jul 27 16:00:21.430298 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430145    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70afda37-1e58-4caa-be32-7e7e7ea3d5ba" containerName="revoke-keys"
Jul 27 16:00:21.430298 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430150    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="70afda37-1e58-4caa-be32-7e7e7ea3d5ba" containerName="revoke-keys"
Jul 27 16:00:21.430298 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430156    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca6ff2bd-f602-43bb-98b4-ba8236256f05" containerName="cleanup"
Jul 27 16:00:21.430298 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430161    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca6ff2bd-f602-43bb-98b4-ba8236256f05" containerName="cleanup"
Jul 27 16:00:21.430298 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430168    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97fa695b-de5e-4165-bf01-374ac513e2e1" containerName="cleanup"
Jul 27 16:00:21.430298 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430172    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="97fa695b-de5e-4165-bf01-374ac513e2e1" containerName="cleanup"
Jul 27 16:00:21.430298 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430227    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="70afda37-1e58-4caa-be32-7e7e7ea3d5ba" containerName="revoke-keys"
Jul 27 16:00:21.430298 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430235    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="01b364c2-3223-4be3-af53-a31b213944a7" containerName="cleanup"
Jul 27 16:00:21.430298 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430241    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca6ff2bd-f602-43bb-98b4-ba8236256f05" containerName="cleanup"
Jul 27 16:00:21.430298 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430249    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="bda50b2a-8bcd-46d3-a4ba-2d17fcbc6442" containerName="maas-api"
Jul 27 16:00:21.430298 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430255    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec83830a-2c36-4b82-b246-c81302d15d53" containerName="cleanup"
Jul 27 16:00:21.430298 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.430260    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="97fa695b-de5e-4165-bf01-374ac513e2e1" containerName="cleanup"
Jul 27 16:00:21.434666 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.434637    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw"
Jul 27 16:00:21.437764 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.437735    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bgfwg\""
Jul 27 16:00:21.440179 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.440155    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw"]
Jul 27 16:00:21.550133 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.550097    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1432f23-5a1a-4297-835f-e7fe7aae052c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw\" (UID: \"d1432f23-5a1a-4297-835f-e7fe7aae052c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw"
Jul 27 16:00:21.650862 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.650830    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1432f23-5a1a-4297-835f-e7fe7aae052c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw\" (UID: \"d1432f23-5a1a-4297-835f-e7fe7aae052c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw"
Jul 27 16:00:21.651415 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.651396    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1432f23-5a1a-4297-835f-e7fe7aae052c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw\" (UID: \"d1432f23-5a1a-4297-835f-e7fe7aae052c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw"
Jul 27 16:00:21.747585 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.747552    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw"
Jul 27 16:00:21.887502 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.887398    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw"]
Jul 27 16:00:21.890342 ip-10-0-143-46 kubenswrapper[2548]: W0727 16:00:21.890316    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1432f23_5a1a_4297_835f_e7fe7aae052c.slice/crio-60aae7cfeb31116f889644213bfcd478ebe2523ab28f2405d5ed711048f70af3 WatchSource:0}: Error finding container 60aae7cfeb31116f889644213bfcd478ebe2523ab28f2405d5ed711048f70af3: Status 404 returned error can't find the container with id 60aae7cfeb31116f889644213bfcd478ebe2523ab28f2405d5ed711048f70af3
Jul 27 16:00:21.938980 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:21.938957    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:22.072920 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:22.072879    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw" event={"ID":"d1432f23-5a1a-4297-835f-e7fe7aae052c","Type":"ContainerStarted","Data":"60aae7cfeb31116f889644213bfcd478ebe2523ab28f2405d5ed711048f70af3"}
Jul 27 16:00:23.080552 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:23.080510    2548 generic.go:356] "Generic (PLEG): container finished" podID="d1432f23-5a1a-4297-835f-e7fe7aae052c" containerID="f8b625507217c68e829c9b90cd14cf1cde692398059f1cb070c6b31741fd9d6e" exitCode=0
Jul 27 16:00:23.080967 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:23.080595    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw" event={"ID":"d1432f23-5a1a-4297-835f-e7fe7aae052c","Type":"ContainerDied","Data":"f8b625507217c68e829c9b90cd14cf1cde692398059f1cb070c6b31741fd9d6e"}
Jul 27 16:00:24.223087 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:24.223058    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw"
Jul 27 16:00:24.377855 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:24.377767    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1432f23-5a1a-4297-835f-e7fe7aae052c-maas-api-service-ca\") pod \"d1432f23-5a1a-4297-835f-e7fe7aae052c\" (UID: \"d1432f23-5a1a-4297-835f-e7fe7aae052c\") "
Jul 27 16:00:24.378156 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:24.378132    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1432f23-5a1a-4297-835f-e7fe7aae052c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d1432f23-5a1a-4297-835f-e7fe7aae052c" (UID: "d1432f23-5a1a-4297-835f-e7fe7aae052c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 16:00:24.479219 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:24.479183    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1432f23-5a1a-4297-835f-e7fe7aae052c-maas-api-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 16:00:25.093254 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:25.093221    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw"
Jul 27 16:00:25.093254 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:25.093248    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw" event={"ID":"d1432f23-5a1a-4297-835f-e7fe7aae052c","Type":"ContainerDied","Data":"60aae7cfeb31116f889644213bfcd478ebe2523ab28f2405d5ed711048f70af3"}
Jul 27 16:00:25.093449 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:25.093298    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60aae7cfeb31116f889644213bfcd478ebe2523ab28f2405d5ed711048f70af3"
Jul 27 16:00:31.857612 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:31.857569    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"]
Jul 27 16:00:31.858213 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:31.857960    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8" podUID="33f33f31-96fe-4603-adac-7dbebcd91e1c" containerName="maas-api" containerID="cri-o://a4c0d4d955da49006534da06952eb30618c7fb78c68dd047c3e44b79271ddd74" gracePeriod=30
Jul 27 16:00:31.886862 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:31.886834    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f58a29-29752800-ghkcq"]
Jul 27 16:00:31.890753 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:31.890722    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f58a29-29752800-ghkcq"]
Jul 27 16:00:31.893835 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:31.893815    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:31.924295 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:31.924257    2548 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8" podUID="33f33f31-96fe-4603-adac-7dbebcd91e1c" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.71:8443/health\": dial tcp 10.133.0.71:8443: connect: connection refused"
Jul 27 16:00:32.115266 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.115184    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 16:00:32.130222 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.130191    2548 generic.go:356] "Generic (PLEG): container finished" podID="33f33f31-96fe-4603-adac-7dbebcd91e1c" containerID="a4c0d4d955da49006534da06952eb30618c7fb78c68dd047c3e44b79271ddd74" exitCode=0
Jul 27 16:00:32.130385 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.130243    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"
Jul 27 16:00:32.130385 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.130271    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8" event={"ID":"33f33f31-96fe-4603-adac-7dbebcd91e1c","Type":"ContainerDied","Data":"a4c0d4d955da49006534da06952eb30618c7fb78c68dd047c3e44b79271ddd74"}
Jul 27 16:00:32.130385 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.130381    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8" event={"ID":"33f33f31-96fe-4603-adac-7dbebcd91e1c","Type":"ContainerDied","Data":"d97eaa95f6ba4891fa3ab18645f04366d012664d08d4294c876394cfa05b3faa"}
Jul 27 16:00:32.130519 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.130410    2548 scope.go:117] "RemoveContainer" containerID="a4c0d4d955da49006534da06952eb30618c7fb78c68dd047c3e44b79271ddd74"
Jul 27 16:00:32.142888 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.142857    2548 scope.go:117] "RemoveContainer" containerID="a4c0d4d955da49006534da06952eb30618c7fb78c68dd047c3e44b79271ddd74"
Jul 27 16:00:32.143245 ip-10-0-143-46 kubenswrapper[2548]: E0727 16:00:32.143216    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4c0d4d955da49006534da06952eb30618c7fb78c68dd047c3e44b79271ddd74\": container with ID starting with a4c0d4d955da49006534da06952eb30618c7fb78c68dd047c3e44b79271ddd74 not found: ID does not exist" containerID="a4c0d4d955da49006534da06952eb30618c7fb78c68dd047c3e44b79271ddd74"
Jul 27 16:00:32.143330 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.143257    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4c0d4d955da49006534da06952eb30618c7fb78c68dd047c3e44b79271ddd74"} err="failed to get container status \"a4c0d4d955da49006534da06952eb30618c7fb78c68dd047c3e44b79271ddd74\": rpc error: code = NotFound desc = could not find container \"a4c0d4d955da49006534da06952eb30618c7fb78c68dd047c3e44b79271ddd74\": container with ID starting with a4c0d4d955da49006534da06952eb30618c7fb78c68dd047c3e44b79271ddd74 not found: ID does not exist"
Jul 27 16:00:32.149654 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.149634    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2fv5\" (UniqueName: \"kubernetes.io/projected/33f33f31-96fe-4603-adac-7dbebcd91e1c-kube-api-access-m2fv5\") pod \"33f33f31-96fe-4603-adac-7dbebcd91e1c\" (UID: \"33f33f31-96fe-4603-adac-7dbebcd91e1c\") "
Jul 27 16:00:32.149756 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.149674    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f33f31-96fe-4603-adac-7dbebcd91e1c-openshift-service-ca\") pod \"33f33f31-96fe-4603-adac-7dbebcd91e1c\" (UID: \"33f33f31-96fe-4603-adac-7dbebcd91e1c\") "
Jul 27 16:00:32.149756 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.149699    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f33f31-96fe-4603-adac-7dbebcd91e1c-maas-api-tls\") pod \"33f33f31-96fe-4603-adac-7dbebcd91e1c\" (UID: \"33f33f31-96fe-4603-adac-7dbebcd91e1c\") "
Jul 27 16:00:32.150043 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.150015    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33f33f31-96fe-4603-adac-7dbebcd91e1c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33f33f31-96fe-4603-adac-7dbebcd91e1c" (UID: "33f33f31-96fe-4603-adac-7dbebcd91e1c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 16:00:32.152037 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.152016    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33f33f31-96fe-4603-adac-7dbebcd91e1c-kube-api-access-m2fv5" (OuterVolumeSpecName: "kube-api-access-m2fv5") pod "33f33f31-96fe-4603-adac-7dbebcd91e1c" (UID: "33f33f31-96fe-4603-adac-7dbebcd91e1c"). InnerVolumeSpecName "kube-api-access-m2fv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 16:00:32.152037 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.152019    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33f33f31-96fe-4603-adac-7dbebcd91e1c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33f33f31-96fe-4603-adac-7dbebcd91e1c" (UID: "33f33f31-96fe-4603-adac-7dbebcd91e1c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 16:00:32.251507 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.251474    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f33f31-96fe-4603-adac-7dbebcd91e1c-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 16:00:32.251507 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.251499    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f33f31-96fe-4603-adac-7dbebcd91e1c-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 16:00:32.251507 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.251509    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2fv5\" (UniqueName: \"kubernetes.io/projected/33f33f31-96fe-4603-adac-7dbebcd91e1c-kube-api-access-m2fv5\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 16:00:32.375877 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.375799    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:32.432985 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.432954    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:32.456055 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.456031    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"]
Jul 27 16:00:32.462353 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:32.462332    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f58a29-57bb88f85b-7w5d8"]
Jul 27 16:00:33.139488 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:33.139461    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33f33f31-96fe-4603-adac-7dbebcd91e1c" path="/var/lib/kubelet/pods/33f33f31-96fe-4603-adac-7dbebcd91e1c/volumes"
Jul 27 16:00:33.139876 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:33.139748    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca6ff2bd-f602-43bb-98b4-ba8236256f05" path="/var/lib/kubelet/pods/ca6ff2bd-f602-43bb-98b4-ba8236256f05/volumes"
Jul 27 16:00:36.492937 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:36.492895    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw"]
Jul 27 16:00:36.497607 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:36.497583    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f58a29-77bdw"]
Jul 27 16:00:36.529297 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:36.529271    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:36.879157 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:36.879081    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:37.139181 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:37.139104    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1432f23-5a1a-4297-835f-e7fe7aae052c" path="/var/lib/kubelet/pods/d1432f23-5a1a-4297-835f-e7fe7aae052c/volumes"
Jul 27 16:00:42.970270 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:42.970230    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:43.613445 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:43.613411    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk"]
Jul 27 16:00:43.613823 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:43.613810    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1432f23-5a1a-4297-835f-e7fe7aae052c" containerName="revoke-keys"
Jul 27 16:00:43.613881 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:43.613824    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1432f23-5a1a-4297-835f-e7fe7aae052c" containerName="revoke-keys"
Jul 27 16:00:43.613881 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:43.613845    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33f33f31-96fe-4603-adac-7dbebcd91e1c" containerName="maas-api"
Jul 27 16:00:43.613881 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:43.613850    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="33f33f31-96fe-4603-adac-7dbebcd91e1c" containerName="maas-api"
Jul 27 16:00:43.613996 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:43.613987    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="33f33f31-96fe-4603-adac-7dbebcd91e1c" containerName="maas-api"
Jul 27 16:00:43.614084 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:43.614070    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1432f23-5a1a-4297-835f-e7fe7aae052c" containerName="revoke-keys"
Jul 27 16:00:43.617259 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:43.617240    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk"
Jul 27 16:00:43.619914 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:43.619889    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bgfwg\""
Jul 27 16:00:43.625143 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:43.624894    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk"]
Jul 27 16:00:43.652646 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:43.652617    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48d9dd92-ca4a-4d19-99a5-8c05560821c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk\" (UID: \"48d9dd92-ca4a-4d19-99a5-8c05560821c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk"
Jul 27 16:00:43.753604 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:43.753555    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48d9dd92-ca4a-4d19-99a5-8c05560821c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk\" (UID: \"48d9dd92-ca4a-4d19-99a5-8c05560821c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk"
Jul 27 16:00:43.754146 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:43.754125    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48d9dd92-ca4a-4d19-99a5-8c05560821c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk\" (UID: \"48d9dd92-ca4a-4d19-99a5-8c05560821c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk"
Jul 27 16:00:43.929211 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:43.929123    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk"
Jul 27 16:00:44.075459 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:44.075427    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk"]
Jul 27 16:00:44.076887 ip-10-0-143-46 kubenswrapper[2548]: W0727 16:00:44.076852    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48d9dd92_ca4a_4d19_99a5_8c05560821c2.slice/crio-d8e6cb9fc193738e954c49f7d0053ee6430d0358d5f8fd08d196cdcf3a91d9c5 WatchSource:0}: Error finding container d8e6cb9fc193738e954c49f7d0053ee6430d0358d5f8fd08d196cdcf3a91d9c5: Status 404 returned error can't find the container with id d8e6cb9fc193738e954c49f7d0053ee6430d0358d5f8fd08d196cdcf3a91d9c5
Jul 27 16:00:44.195369 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:44.195285    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk" event={"ID":"48d9dd92-ca4a-4d19-99a5-8c05560821c2","Type":"ContainerStarted","Data":"d8e6cb9fc193738e954c49f7d0053ee6430d0358d5f8fd08d196cdcf3a91d9c5"}
Jul 27 16:00:44.976915 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:44.976884    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:45.202308 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:45.202276    2548 generic.go:356] "Generic (PLEG): container finished" podID="48d9dd92-ca4a-4d19-99a5-8c05560821c2" containerID="59ec5eaaa2c5e9add6271554ebab741c60986301019ed9771131d657c6a9f07e" exitCode=0
Jul 27 16:00:45.202704 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:45.202342    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk" event={"ID":"48d9dd92-ca4a-4d19-99a5-8c05560821c2","Type":"ContainerDied","Data":"59ec5eaaa2c5e9add6271554ebab741c60986301019ed9771131d657c6a9f07e"}
Jul 27 16:00:46.361081 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:46.361055    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk"
Jul 27 16:00:46.477295 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:46.477272    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48d9dd92-ca4a-4d19-99a5-8c05560821c2-maas-api-service-ca\") pod \"48d9dd92-ca4a-4d19-99a5-8c05560821c2\" (UID: \"48d9dd92-ca4a-4d19-99a5-8c05560821c2\") "
Jul 27 16:00:46.477629 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:46.477607    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48d9dd92-ca4a-4d19-99a5-8c05560821c2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "48d9dd92-ca4a-4d19-99a5-8c05560821c2" (UID: "48d9dd92-ca4a-4d19-99a5-8c05560821c2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 16:00:46.578079 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:46.578051    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48d9dd92-ca4a-4d19-99a5-8c05560821c2-maas-api-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 16:00:47.213554 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:47.213508    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk"
Jul 27 16:00:47.213554 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:47.213516    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk" event={"ID":"48d9dd92-ca4a-4d19-99a5-8c05560821c2","Type":"ContainerDied","Data":"d8e6cb9fc193738e954c49f7d0053ee6430d0358d5f8fd08d196cdcf3a91d9c5"}
Jul 27 16:00:47.213746 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:47.213581    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d8e6cb9fc193738e954c49f7d0053ee6430d0358d5f8fd08d196cdcf3a91d9c5"
Jul 27 16:00:54.020495 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.020304    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"]
Jul 27 16:00:54.021221 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.021161    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt" podUID="85576123-98c8-413d-88b7-d386d850d444" containerName="maas-api" containerID="cri-o://fc2e11a39e93f8a6cc4da6a985938a38036a97f584ee1504e1fb7f3d6da75a64" gracePeriod=30
Jul 27 16:00:54.046386 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.046348    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:54.046554 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.046356    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-052d16-29752800-9gjwn"]
Jul 27 16:00:54.049010 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.048982    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-052d16-29752800-9gjwn"]
Jul 27 16:00:54.143796 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.143764    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:54.245171 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.245130    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:54.252834 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.252804    2548 generic.go:356] "Generic (PLEG): container finished" podID="85576123-98c8-413d-88b7-d386d850d444" containerID="fc2e11a39e93f8a6cc4da6a985938a38036a97f584ee1504e1fb7f3d6da75a64" exitCode=0
Jul 27 16:00:54.253048 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.252920    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt" event={"ID":"85576123-98c8-413d-88b7-d386d850d444","Type":"ContainerDied","Data":"fc2e11a39e93f8a6cc4da6a985938a38036a97f584ee1504e1fb7f3d6da75a64"}
Jul 27 16:00:54.280257 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.280204    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 16:00:54.354164 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.354134    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85576123-98c8-413d-88b7-d386d850d444-maas-api-tls\") pod \"85576123-98c8-413d-88b7-d386d850d444\" (UID: \"85576123-98c8-413d-88b7-d386d850d444\") "
Jul 27 16:00:54.354334 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.354201    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85576123-98c8-413d-88b7-d386d850d444-openshift-service-ca\") pod \"85576123-98c8-413d-88b7-d386d850d444\" (UID: \"85576123-98c8-413d-88b7-d386d850d444\") "
Jul 27 16:00:54.354334 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.354295    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n65mn\" (UniqueName: \"kubernetes.io/projected/85576123-98c8-413d-88b7-d386d850d444-kube-api-access-n65mn\") pod \"85576123-98c8-413d-88b7-d386d850d444\" (UID: \"85576123-98c8-413d-88b7-d386d850d444\") "
Jul 27 16:00:54.354693 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.354587    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85576123-98c8-413d-88b7-d386d850d444-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "85576123-98c8-413d-88b7-d386d850d444" (UID: "85576123-98c8-413d-88b7-d386d850d444"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 16:00:54.356572 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.356527    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85576123-98c8-413d-88b7-d386d850d444-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "85576123-98c8-413d-88b7-d386d850d444" (UID: "85576123-98c8-413d-88b7-d386d850d444"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 16:00:54.356771 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.356750    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85576123-98c8-413d-88b7-d386d850d444-kube-api-access-n65mn" (OuterVolumeSpecName: "kube-api-access-n65mn") pod "85576123-98c8-413d-88b7-d386d850d444" (UID: "85576123-98c8-413d-88b7-d386d850d444"). InnerVolumeSpecName "kube-api-access-n65mn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 16:00:54.455283 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.455238    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n65mn\" (UniqueName: \"kubernetes.io/projected/85576123-98c8-413d-88b7-d386d850d444-kube-api-access-n65mn\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 16:00:54.455283 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.455276    2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85576123-98c8-413d-88b7-d386d850d444-maas-api-tls\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 16:00:54.455283 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:54.455286    2548 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85576123-98c8-413d-88b7-d386d850d444-openshift-service-ca\") on node \"ip-10-0-143-46.ec2.internal\" DevicePath \"\""
Jul 27 16:00:55.139478 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:55.139434    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01b364c2-3223-4be3-af53-a31b213944a7" path="/var/lib/kubelet/pods/01b364c2-3223-4be3-af53-a31b213944a7/volumes"
Jul 27 16:00:55.259167 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:55.259135    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt" event={"ID":"85576123-98c8-413d-88b7-d386d850d444","Type":"ContainerDied","Data":"f354cf38f1030ac089560d9cbbd63a0a0b1d99105ae1dc4733084bc7bf8798b5"}
Jul 27 16:00:55.259307 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:55.259174    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"
Jul 27 16:00:55.259307 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:55.259212    2548 scope.go:117] "RemoveContainer" containerID="fc2e11a39e93f8a6cc4da6a985938a38036a97f584ee1504e1fb7f3d6da75a64"
Jul 27 16:00:55.277806 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:55.277781    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"]
Jul 27 16:00:55.281352 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:55.281332    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-052d16-5ccc78c78-d77zt"]
Jul 27 16:00:57.139771 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:57.139732    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85576123-98c8-413d-88b7-d386d850d444" path="/var/lib/kubelet/pods/85576123-98c8-413d-88b7-d386d850d444/volumes"
Jul 27 16:00:58.678393 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:58.678353    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk"]
Jul 27 16:00:58.681081 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:58.681041    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:58.681220 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:58.681142    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-052d16-z6zdk"]
Jul 27 16:00:59.078947 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:59.078899    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:00:59.138980 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:00:59.138948    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48d9dd92-ca4a-4d19-99a5-8c05560821c2" path="/var/lib/kubelet/pods/48d9dd92-ca4a-4d19-99a5-8c05560821c2/volumes"
Jul 27 16:01:05.297114 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:05.297079    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:01:05.377325 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:05.377295    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:01:05.570038 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:05.569965    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:01:05.825477 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:05.825395    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:01:12.051401 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:12.051367    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77cf85d6c8-bm466_403e2717-a834-46a4-b36a-29e220418ad9/manager/0.log"
Jul 27 16:01:12.169758 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:12.169728    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-wj86m_d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8/server/0.log"
Jul 27 16:01:12.682852 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:12.682824    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78959f89bb-tr8hk_6516aaf3-43f8-43a7-80c3-72a411ed9497/copy-manifests/0.log"
Jul 27 16:01:12.690269 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:12.690243    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78959f89bb-tr8hk_6516aaf3-43f8-43a7-80c3-72a411ed9497/manager/0.log"
Jul 27 16:01:13.039014 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:13.038973    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-574b54f485-9z998_2ed8508b-568f-45f5-936c-eee83fe7734b/maas-api/0.log"
Jul 27 16:01:13.265000 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:13.264970    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29752785-6k5gq_a9442ae1-7269-45e9-8039-2a1e1e75216f/cleanup/0.log"
Jul 27 16:01:13.382860 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:13.382775    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29752800-w8dph_97fa695b-de5e-4165-bf01-374ac513e2e1/cleanup/0.log"
Jul 27 16:01:13.494723 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:13.494686    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-vz5dv_6ee67b26-8e48-4370-9a20-223b9238e72f/postgres/0.log"
Jul 27 16:01:14.924219 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:14.924189    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-bdrwf_35c40fff-5ea1-4320-ba36-88b589161de8/manager/0.log"
Jul 27 16:01:15.036151 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:15.036121    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-2lf8g_ed328f85-503e-4671-83e9-b44f78f30568/manager/1.log"
Jul 27 16:01:15.268056 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:15.268022    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-wn4ns_8bc7fa86-8fc8-4efd-8bf1-5b1fdd669742/registry-server/0.log"
Jul 27 16:01:15.400781 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:15.400741    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jqpfg_0eec5852-d33e-40f7-8d1f-43e7d7eead44/manager/0.log"
Jul 27 16:01:15.643410 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:15.643326    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-mkg6b_714e3d16-6c58-474f-bab1-d1ef2368096c/manager/0.log"
Jul 27 16:01:16.144327 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:16.144295    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fgbbp_dc21e594-c8be-4779-b147-9c0f062cb697/discovery/0.log"
Jul 27 16:01:16.392338 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:16.392300    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cd65dd67b-xsqns_df921f0f-6513-4002-9a3c-da80931d58a4/kube-auth-proxy/0.log"
Jul 27 16:01:16.648190 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:16.648157    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-hsfch_4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8/payload-pre-processing/0.log"
Jul 27 16:01:17.255607 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:17.255580    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-64w4q_04d3db51-1ae9-4e54-816c-c86f55b7d7d7/main/0.log"
Jul 27 16:01:17.489262 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:17.489231    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-7444p_bc6468e1-d6e0-4fdd-bb81-1b88d5216910/main/0.log"
Jul 27 16:01:17.719184 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:17.719076    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-cd8tx_c2c26f9f-92dd-447c-a388-8078c36b9b17/main/0.log"
Jul 27 16:01:23.499423 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:23.499383    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:01:23.595478 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:23.595447    2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log" limit=1024
Jul 27 16:01:29.660686 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:29.660662    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-njr4n_f1023b4e-580a-4dc0-8a69-4167ae7cec34/global-pull-secret-syncer/0.log"
Jul 27 16:01:29.710661 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:29.710637    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8hhwl_63d99334-3633-4f35-9097-385b6aed074b/konnectivity-agent/0.log"
Jul 27 16:01:29.883788 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:29.883756    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-46.ec2.internal_7c7c4cbd4197f801582237d506698c93/haproxy/0.log"
Jul 27 16:01:35.367280 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:35.367232    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-bdrwf_35c40fff-5ea1-4320-ba36-88b589161de8/manager/0.log"
Jul 27 16:01:35.395286 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:35.395246    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-2lf8g_ed328f85-503e-4671-83e9-b44f78f30568/manager/0.log"
Jul 27 16:01:35.395458 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:35.395371    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-2lf8g_ed328f85-503e-4671-83e9-b44f78f30568/manager/1.log"
Jul 27 16:01:35.462146 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:35.462112    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-wn4ns_8bc7fa86-8fc8-4efd-8bf1-5b1fdd669742/registry-server/0.log"
Jul 27 16:01:35.602859 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:35.602779    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jqpfg_0eec5852-d33e-40f7-8d1f-43e7d7eead44/manager/0.log"
Jul 27 16:01:35.715447 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:35.715408    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-mkg6b_714e3d16-6c58-474f-bab1-d1ef2368096c/manager/0.log"
Jul 27 16:01:38.095147 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.095117    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-9455d7f8-4lfdc_c851e1cb-42ea-453f-a7e8-2735be4fc62a/metrics-server/0.log"
Jul 27 16:01:38.125789 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.125759    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-7xbtq_8ad810d0-ab4c-4cbc-a086-4ddbc5de88e7/monitoring-plugin/0.log"
Jul 27 16:01:38.158457 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.158422    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gdl67_120ac523-537f-4733-8449-86d168cba4f7/node-exporter/0.log"
Jul 27 16:01:38.180831 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.180800    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gdl67_120ac523-537f-4733-8449-86d168cba4f7/kube-rbac-proxy/0.log"
Jul 27 16:01:38.206421 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.206394    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gdl67_120ac523-537f-4733-8449-86d168cba4f7/init-textfile/0.log"
Jul 27 16:01:38.461979 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.461903    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-cc5lw_6cd26152-a496-4c0d-ae7c-040a5b44cb56/kube-rbac-proxy-main/0.log"
Jul 27 16:01:38.491372 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.491344    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-cc5lw_6cd26152-a496-4c0d-ae7c-040a5b44cb56/kube-rbac-proxy-self/0.log"
Jul 27 16:01:38.514435 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.514410    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-cc5lw_6cd26152-a496-4c0d-ae7c-040a5b44cb56/openshift-state-metrics/0.log"
Jul 27 16:01:38.550474 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.550449    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1ad3e603-6239-4042-ae14-8a788a6c757a/prometheus/0.log"
Jul 27 16:01:38.574823 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.574797    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1ad3e603-6239-4042-ae14-8a788a6c757a/config-reloader/0.log"
Jul 27 16:01:38.601680 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.601652    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1ad3e603-6239-4042-ae14-8a788a6c757a/thanos-sidecar/0.log"
Jul 27 16:01:38.626210 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.626173    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1ad3e603-6239-4042-ae14-8a788a6c757a/kube-rbac-proxy-web/0.log"
Jul 27 16:01:38.654339 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.654311    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1ad3e603-6239-4042-ae14-8a788a6c757a/kube-rbac-proxy/0.log"
Jul 27 16:01:38.681183 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.681150    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1ad3e603-6239-4042-ae14-8a788a6c757a/kube-rbac-proxy-thanos/0.log"
Jul 27 16:01:38.714209 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.714147    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1ad3e603-6239-4042-ae14-8a788a6c757a/init-config-reloader/0.log"
Jul 27 16:01:38.881044 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.881012    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85b764cd7f-8cfkc_12742c2c-05be-4268-a2f6-e367af8aa791/telemeter-client/0.log"
Jul 27 16:01:38.904185 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.904156    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85b764cd7f-8cfkc_12742c2c-05be-4268-a2f6-e367af8aa791/reload/0.log"
Jul 27 16:01:38.927925 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:38.927906    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85b764cd7f-8cfkc_12742c2c-05be-4268-a2f6-e367af8aa791/kube-rbac-proxy/0.log"
Jul 27 16:01:41.345082 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.345054    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-64dc66b447-sr8n8_4171643d-b939-4b6f-8d90-dc3b18f7f9c3/console/0.log"
Jul 27 16:01:41.506684 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.506650    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"]
Jul 27 16:01:41.507221 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.507190    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85576123-98c8-413d-88b7-d386d850d444" containerName="maas-api"
Jul 27 16:01:41.507221 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.507218    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="85576123-98c8-413d-88b7-d386d850d444" containerName="maas-api"
Jul 27 16:01:41.507395 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.507267    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48d9dd92-ca4a-4d19-99a5-8c05560821c2" containerName="revoke-keys"
Jul 27 16:01:41.507395 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.507276    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="48d9dd92-ca4a-4d19-99a5-8c05560821c2" containerName="revoke-keys"
Jul 27 16:01:41.507395 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.507343    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="85576123-98c8-413d-88b7-d386d850d444" containerName="maas-api"
Jul 27 16:01:41.507395 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.507353    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="48d9dd92-ca4a-4d19-99a5-8c05560821c2" containerName="revoke-keys"
Jul 27 16:01:41.510406 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.510387    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.512884 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.512847    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6fjkv\"/\"openshift-service-ca.crt\""
Jul 27 16:01:41.513081 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.513068    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6fjkv\"/\"kube-root-ca.crt\""
Jul 27 16:01:41.514146 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.514132    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6fjkv\"/\"default-dockercfg-65s6n\""
Jul 27 16:01:41.518691 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.518669    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"]
Jul 27 16:01:41.596857 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.596760    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62588e64-5310-4aec-a864-3aa55e413022-lib-modules\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.596857 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.596822    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-976mm\" (UniqueName: \"kubernetes.io/projected/62588e64-5310-4aec-a864-3aa55e413022-kube-api-access-976mm\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.597078 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.596931    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/62588e64-5310-4aec-a864-3aa55e413022-proc\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.597078 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.596964    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62588e64-5310-4aec-a864-3aa55e413022-sys\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.597078 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.596990    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/62588e64-5310-4aec-a864-3aa55e413022-podres\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.698173 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.698138    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-976mm\" (UniqueName: \"kubernetes.io/projected/62588e64-5310-4aec-a864-3aa55e413022-kube-api-access-976mm\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.698386 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.698209    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/62588e64-5310-4aec-a864-3aa55e413022-proc\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.698386 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.698233    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62588e64-5310-4aec-a864-3aa55e413022-sys\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.698386 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.698259    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/62588e64-5310-4aec-a864-3aa55e413022-podres\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.698386 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.698320    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62588e64-5310-4aec-a864-3aa55e413022-lib-modules\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.698386 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.698350    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/62588e64-5310-4aec-a864-3aa55e413022-proc\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.698386 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.698356    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62588e64-5310-4aec-a864-3aa55e413022-sys\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.698674 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.698454    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/62588e64-5310-4aec-a864-3aa55e413022-podres\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.698674 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.698497    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62588e64-5310-4aec-a864-3aa55e413022-lib-modules\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.709009 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.708988    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-976mm\" (UniqueName: \"kubernetes.io/projected/62588e64-5310-4aec-a864-3aa55e413022-kube-api-access-976mm\") pod \"perf-node-gather-daemonset-ws2q9\" (UID: \"62588e64-5310-4aec-a864-3aa55e413022\") " pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.821010 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.820971    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:41.954124 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:41.954099    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"]
Jul 27 16:01:41.956306 ip-10-0-143-46 kubenswrapper[2548]: W0727 16:01:41.956276    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod62588e64_5310_4aec_a864_3aa55e413022.slice/crio-ca38e31b9d50198325282fbc05cf7a9f1e287992b0a0fa6e0834aa6e1cfafc89 WatchSource:0}: Error finding container ca38e31b9d50198325282fbc05cf7a9f1e287992b0a0fa6e0834aa6e1cfafc89: Status 404 returned error can't find the container with id ca38e31b9d50198325282fbc05cf7a9f1e287992b0a0fa6e0834aa6e1cfafc89
Jul 27 16:01:42.505855 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:42.505808    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9" event={"ID":"62588e64-5310-4aec-a864-3aa55e413022","Type":"ContainerStarted","Data":"5e9afb0a0f514c5ffa0e2259bd3f0f2cb0826fc9c95e5b7689c1220bae72fdee"}
Jul 27 16:01:42.506241 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:42.505878    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9" event={"ID":"62588e64-5310-4aec-a864-3aa55e413022","Type":"ContainerStarted","Data":"ca38e31b9d50198325282fbc05cf7a9f1e287992b0a0fa6e0834aa6e1cfafc89"}
Jul 27 16:01:42.506241 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:42.505918    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:42.525234 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:42.525185    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9" podStartSLOduration=1.525171249 podStartE2EDuration="1.525171249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 16:01:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:01:42.523385848 +0000 UTC m=+3373.952061032" watchObservedRunningTime="2026-07-27 16:01:42.525171249 +0000 UTC m=+3373.953846432"
Jul 27 16:01:43.219272 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:43.219240    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zrg2s_752e4bdb-fc3f-4c7f-8c7f-04edea561071/dns/0.log"
Jul 27 16:01:43.239491 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:43.239465    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zrg2s_752e4bdb-fc3f-4c7f-8c7f-04edea561071/kube-rbac-proxy/0.log"
Jul 27 16:01:43.262345 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:43.262322    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-52jxv_78b76a57-4e6c-4189-9225-d4e032894ca2/dns-node-resolver/0.log"
Jul 27 16:01:43.906314 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:43.906284    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bgwms_c3253118-4914-4dc8-a105-d30a1f850364/node-ca/0.log"
Jul 27 16:01:45.071905 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:45.071873    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fgbbp_dc21e594-c8be-4779-b147-9c0f062cb697/discovery/0.log"
Jul 27 16:01:45.119228 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:45.119200    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cd65dd67b-xsqns_df921f0f-6513-4002-9a3c-da80931d58a4/kube-auth-proxy/0.log"
Jul 27 16:01:45.215743 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:45.215715    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-hsfch_4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8/payload-pre-processing/0.log"
Jul 27 16:01:46.144110 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:46.144084    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wnvb6_46e7ac8e-6e47-48e1-bce2-20f3897c3af8/serve-healthcheck-canary/0.log"
Jul 27 16:01:46.651223 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:46.651191    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dnp8b_3962639b-3a18-4284-9454-f14e7c24f8a3/kube-rbac-proxy/0.log"
Jul 27 16:01:46.672288 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:46.672265    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dnp8b_3962639b-3a18-4284-9454-f14e7c24f8a3/exporter/0.log"
Jul 27 16:01:46.693508 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:46.693484    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dnp8b_3962639b-3a18-4284-9454-f14e7c24f8a3/extractor/0.log"
Jul 27 16:01:48.521099 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:48.521071    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6fjkv/perf-node-gather-daemonset-ws2q9"
Jul 27 16:01:49.406652 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:49.406624    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77cf85d6c8-bm466_403e2717-a834-46a4-b36a-29e220418ad9/manager/0.log"
Jul 27 16:01:49.428809 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:49.428789    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-wj86m_d6dfb4ad-bc0b-4d17-a659-bfad5bf07ed8/server/0.log"
Jul 27 16:01:49.735795 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:49.735764    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78959f89bb-tr8hk_6516aaf3-43f8-43a7-80c3-72a411ed9497/manager/0.log"
Jul 27 16:01:49.757981 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:49.757962    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78959f89bb-tr8hk_6516aaf3-43f8-43a7-80c3-72a411ed9497/copy-manifests/0.log"
Jul 27 16:01:50.213118 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:50.213088    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fgbbp_dc21e594-c8be-4779-b147-9c0f062cb697/discovery/0.log"
Jul 27 16:01:50.240549 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:50.240518    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cd65dd67b-xsqns_df921f0f-6513-4002-9a3c-da80931d58a4/kube-auth-proxy/0.log"
Jul 27 16:01:50.337484 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:50.337451    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-hsfch_4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8/payload-pre-processing/0.log"
Jul 27 16:01:52.177268 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:52.177190    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fgbbp_dc21e594-c8be-4779-b147-9c0f062cb697/discovery/0.log"
Jul 27 16:01:52.225694 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:52.225665    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cd65dd67b-xsqns_df921f0f-6513-4002-9a3c-da80931d58a4/kube-auth-proxy/0.log"
Jul 27 16:01:52.313472 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:52.313443    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-hsfch_4ca7b7fd-d93a-4c9c-bef7-438f6b9249e8/payload-pre-processing/0.log"
Jul 27 16:01:58.002648 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:58.002614    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7z4g4_92215e75-de40-412c-85a4-0eabf00f5dc1/kube-multus-additional-cni-plugins/0.log"
Jul 27 16:01:58.028505 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:58.028475    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7z4g4_92215e75-de40-412c-85a4-0eabf00f5dc1/egress-router-binary-copy/0.log"
Jul 27 16:01:58.054516 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:58.054485    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7z4g4_92215e75-de40-412c-85a4-0eabf00f5dc1/cni-plugins/0.log"
Jul 27 16:01:58.083860 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:58.083839    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7z4g4_92215e75-de40-412c-85a4-0eabf00f5dc1/bond-cni-plugin/0.log"
Jul 27 16:01:58.115109 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:58.115088    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7z4g4_92215e75-de40-412c-85a4-0eabf00f5dc1/routeoverride-cni/0.log"
Jul 27 16:01:58.142561 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:58.142510    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7z4g4_92215e75-de40-412c-85a4-0eabf00f5dc1/whereabouts-cni-bincopy/0.log"
Jul 27 16:01:58.171887 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:58.171862    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7z4g4_92215e75-de40-412c-85a4-0eabf00f5dc1/whereabouts-cni/0.log"
Jul 27 16:01:58.772026 ip-10-0-143-46 kubenswrapper[2548]: I0727 16:01:58.772000    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gcq99_e9ebfe61-7a6e-4c9f-9437-0096cb7b7531/kube-multus/0.log"