Jul 30 03:45:00.834836 ip-10-0-136-104 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 03:45:01.268316 ip-10-0-136-104 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 03:45:01.268316 ip-10-0-136-104 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 03:45:01.268316 ip-10-0-136-104 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 03:45:01.268316 ip-10-0-136-104 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 03:45:01.268316 ip-10-0-136-104 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 03:45:01.269173 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.269085    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 03:45:01.275302 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275286    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 03:45:01.275302 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275299    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 03:45:01.275302 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275303    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275306    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275309    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275311    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275314    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275317    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275319    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275322    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275324    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275327    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275329    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275332    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275334    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275336    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275339    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275341    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275343    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275346    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275348    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 03:45:01.275427 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275353    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275357    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275361    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275364    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275367    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275370    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275372    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275375    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275377    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275379    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275382    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275384    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275387    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275389    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275391    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275395    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275398    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275401    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275403    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275406    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 03:45:01.275884 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275409    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275411    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275414    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275416    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275419    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275421    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275423    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275427    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275431    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275434    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275438    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275441    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275443    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275446    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275449    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275451    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275453    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275456    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275460    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 03:45:01.276370 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275464    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275466    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275469    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275471    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275474    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275476    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275478    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275481    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275485    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275487    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275490    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275493    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275496    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275499    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275501    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275503    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275506    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275508    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275511    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275514    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 03:45:01.276841 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275516    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275518    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275522    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275524    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275527    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275542    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275545    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275548    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275550    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275553    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275555    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275558    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275560    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275563    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275565    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275568    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275570    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275573    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.275575    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 03:45:01.277321 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276280    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276287    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276291    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276294    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276297    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276300    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276302    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276305    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276307    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276310    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276312    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276315    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276317    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276320    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276322    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276324    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276327    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276330    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276332    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276335    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 03:45:01.277789 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276337    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276339    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276342    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276344    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276347    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276349    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276351    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276354    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276357    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276359    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276362    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276365    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276367    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276370    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276373    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276376    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276378    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276381    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276384    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276387    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 03:45:01.278307 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276390    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276392    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276394    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276397    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276400    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276402    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276404    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276407    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276409    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276412    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276415    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276417    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276420    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276423    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276426    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276428    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276431    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276433    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276436    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276439    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 03:45:01.278804 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276441    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276443    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276446    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276450    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276453    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276457    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276461    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276464    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276467    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276469    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276473    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276476    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276479    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276481    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276484    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276487    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276489    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276491    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276494    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 03:45:01.279299 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276496    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276499    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276502    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276505    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276508    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276510    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276513    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276515    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276518    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276521    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276523    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276544    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276547    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276549    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276552    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276555    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276557    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276560    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276562    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 03:45:01.279775 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276564    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276965    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276970    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276972    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276975    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276978    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276981    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276983    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276986    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276989    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276991    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276993    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276996    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.276998    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277001    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277004    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277006    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277008    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277011    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277013    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 03:45:01.280227 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277016    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277018    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277021    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277023    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277028    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277031    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277033    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277035    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277038    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277041    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277043    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277046    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277048    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277050    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277053    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277055    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277058    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277060    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277063    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277065    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 03:45:01.280805 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277068    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277072    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277075    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277078    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277081    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277083    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277087    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277092    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277094    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277097    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277099    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277102    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277104    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277107    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277109    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277111    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277116    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277119    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277121    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 03:45:01.281287 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277124    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277126    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277129    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277131    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277134    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277136    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277139    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277142    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277146    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277148    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277151    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277153    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277157    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277159    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277162    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277164    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277167    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277169    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277171    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 03:45:01.281750 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277174    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277178    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277180    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277182    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277185    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277189    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277193    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277196    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277198    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277201    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277204    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277207    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277209    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277212    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277214    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277217    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277219    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277222    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277224    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277227    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 03:45:01.282208 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277229    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277232    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277298    2569 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277305    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277311    2569 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277316    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277320    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277323    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277327    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277332    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277335    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277338    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277341    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277346    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277349    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277352    2569 flags.go:64] FLAG: --cgroup-root=""
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277354    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277357    2569 flags.go:64] FLAG: --client-ca-file=""
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277360    2569 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277363    2569 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277367    2569 flags.go:64] FLAG: --cluster-domain=""
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277369    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277373    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277376    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 03:45:01.282692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277379    2569 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277383    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277385    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277388    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277392    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277394    2569 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277397    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277400    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277403    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277406    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277410    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277413    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277417    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277419    2569 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277423    2569 flags.go:64] FLAG: --enable-server="true"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277426    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277431    2569 flags.go:64] FLAG: --event-burst="100"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277434    2569 flags.go:64] FLAG: --event-qps="50"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277436    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277439    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277442    2569 flags.go:64] FLAG: --eviction-hard=""
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277446    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277449    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277453    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277455    2569 flags.go:64] FLAG: --eviction-soft=""
Jul 30 03:45:01.283304 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277458    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277461    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277464    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277467    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277470    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277473    2569 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277476    2569 flags.go:64] FLAG: --feature-gates=""
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277480    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277483    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277486    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277489    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277491    2569 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277494    2569 flags.go:64] FLAG: --help="false"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277497    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277500    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277502    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277505    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277508    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277512    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277515    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277517    2569 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277520    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277523    2569 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 03:45:01.283905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277526    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277545    2569 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277548    2569 flags.go:64] FLAG: --kube-reserved=""
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277551    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277554    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277558    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277561    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277563    2569 flags.go:64] FLAG: --lock-file=""
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277566    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277569    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277572    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277577    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277580    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277582    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277585    2569 flags.go:64] FLAG: --logging-format="text"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277588    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277591    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277594    2569 flags.go:64] FLAG: --manifest-url=""
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277596    2569 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277600    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277603    2569 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277607    2569 flags.go:64] FLAG: --max-pods="110"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277609    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277612    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277615    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 03:45:01.284485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277618    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277621    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277623    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277626    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277634    2569 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277637    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277639    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277642    2569 flags.go:64] FLAG: --pod-cidr=""
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277645    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277651    2569 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277653    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277657    2569 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277660    2569 flags.go:64] FLAG: --port="10250"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277663    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277666    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e3a2030b168cd11c"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277669    2569 flags.go:64] FLAG: --qos-reserved=""
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277672    2569 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277675    2569 flags.go:64] FLAG: --register-node="true"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277677    2569 flags.go:64] FLAG: --register-with-taints=""
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277681    2569 flags.go:64] FLAG: --registry-burst="10"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277683    2569 flags.go:64] FLAG: --registry-qps="5"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277686    2569 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277689    2569 flags.go:64] FLAG: --reserved-memory=""
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277692    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277695    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 03:45:01.285091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277698    2569 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277700    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277703    2569 flags.go:64] FLAG: --runonce="false"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277706    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277708    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277711    2569 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277714    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277716    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277719    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277722    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277725    2569 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277727    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277730    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277732    2569 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277735    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277738    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277741    2569 flags.go:64] FLAG: --system-cgroups=""
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277743    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277748    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277752    2569 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277755    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277758    2569 flags.go:64] FLAG: --tls-min-version=""
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277761    2569 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277764    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277766    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 03:45:01.285706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277770    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277773    2569 flags.go:64] FLAG: --v="2"
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277777    2569 flags.go:64] FLAG: --version="false"
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277781    2569 flags.go:64] FLAG: --vmodule=""
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277784    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.277787    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277906    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277910    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277913    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277916    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277919    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277921    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277924    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277926    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277929    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277931    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277933    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277936    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277939    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277941    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277944    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 03:45:01.286305 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277946    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277949    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277951    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277956    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277959    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277964    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277966    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277969    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277972    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277974    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277978    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277983    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277986    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277989    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277991    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277995    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.277998    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278001    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278003    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 03:45:01.286852 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278006    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278009    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278011    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278014    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278016    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278019    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278021    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278023    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278026    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278028    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278031    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278033    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278036    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278038    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278041    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278043    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278045    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278048    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278052    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 03:45:01.287312 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278054    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278056    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278059    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278061    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278063    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278067    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278069    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278072    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278074    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278076    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278079    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278081    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278084    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278087    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278089    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278092    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278094    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278097    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278099    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278101    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 03:45:01.287788 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278104    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278106    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278109    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278111    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278113    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278116    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278118    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278120    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278123    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278125    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278127    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278132    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278134    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278137    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278139    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278141    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278144    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278147    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278150    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278152    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 03:45:01.288263 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278155    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278157    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278160    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278162    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278165    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.278167    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.278173    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.286364    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.286378    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.286399    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.286404    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286455    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286460    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286464    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286466    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 03:45:01.288749 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286469    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286472    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286475    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286477    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286480    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286483    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286485    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286488    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286490    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286492    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286495    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286497    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286500    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286502    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286505    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286507    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286510    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286513    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286516    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286518    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 03:45:01.289112 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286521    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286524    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286526    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286552    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286557    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286561    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286564    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286566    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286568    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286571    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286573    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286576    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286580    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286584    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286587    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286589    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286592    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286594    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286597    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 03:45:01.289608 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286599    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286602    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286605    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286607    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286610    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286612    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286615    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286617    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286620    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286622    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286626    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286630    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286632    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286635    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286638    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286640    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286643    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286645    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286650    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 03:45:01.290055 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286654    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286657    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286660    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286663    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286666    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286669    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286671    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286674    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286676    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286679    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286681    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286684    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286687    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286689    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286692    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286694    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286697    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286699    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286702    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286704    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 03:45:01.290521 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286706    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286709    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286712    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286714    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286717    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286719    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286722    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286724    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286726    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286729    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286731    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286734    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286737    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286740    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286742    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286744    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 03:45:01.291013 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286747    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.286752    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286856    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286860    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286863    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286866    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286869    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286872    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286874    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286877    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286879    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286882    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286884    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286887    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286889    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 03:45:01.291407 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286892    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286895    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286898    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286900    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286902    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286905    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286908    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286912    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286915    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286917    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286920    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286922    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286925    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286928    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286931    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286934    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286936    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286940    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286943    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 03:45:01.291792 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286947    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286949    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286952    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286954    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286957    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286960    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286962    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286965    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286967    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286970    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286972    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286974    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286977    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286979    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286982    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286984    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286987    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286989    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286991    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286994    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 03:45:01.292243 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286996    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.286998    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287001    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287003    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287005    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287009    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287012    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287015    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287018    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287021    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287024    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287026    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287029    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287031    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287033    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287036    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287038    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287041    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287043    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 03:45:01.292812 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287046    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287048    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287050    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287053    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287055    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287057    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287060    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287062    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287064    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287067    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287069    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287071    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287074    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287076    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287079    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287081    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287083    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287086    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287096    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287099    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 03:45:01.293261 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287101    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 03:45:01.293740 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287104    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 03:45:01.293740 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287106    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 03:45:01.293740 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287109    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 03:45:01.293740 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287111    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 03:45:01.293740 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287114    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 03:45:01.293740 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287116    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 03:45:01.293740 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:01.287119    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 03:45:01.293740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.287123    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 03:45:01.293740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.287228    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 03:45:01.293740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.289145    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 03:45:01.293740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.290509    2569 server.go:1019] "Starting client certificate rotation"
Jul 30 03:45:01.293740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.290621    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 03:45:01.293740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.290658    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 03:45:01.294069 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.293100    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 03:45:01.294231 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.294213    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 03:45:01.309206 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.309182    2569 log.go:25] "Validated CRI v1 runtime API"
Jul 30 03:45:01.311139 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.311122    2569 log.go:25] "Validated CRI v1 image API"
Jul 30 03:45:01.312036 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.312021    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 03:45:01.315305 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.315288    2569 fs.go:135] Filesystem UUIDs: map[7817b7a4-fc22-49da-8137-5d7c766bb39c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d4a7ee0c-134e-4fde-9d6c-93c5869933f6:/dev/nvme0n1p4]
Jul 30 03:45:01.315359 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.315305    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 03:45:01.321876 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.321756    2569 manager.go:216] Machine: {Timestamp:2026-07-30 03:45:01.319012126 +0000 UTC m=+0.357723136 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097940 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2719d7c48b128622174f96b2c7f3c9 SystemUUID:ec2719d7-c48b-1286-2217-4f96b2c7f3c9 BootID:e37c4943-47b1-4aba-9495-d44b2b594860 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9b:cb:8c:ec:85 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9b:cb:8c:ec:85 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:8c:27:42:20:7b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 03:45:01.321876 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.321864    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 03:45:01.322018 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.321943    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 03:45:01.322365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.322345    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 03:45:01.324787 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.324756    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 03:45:01.324934 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.324788    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-104.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 03:45:01.325674 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.325664    2569 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 03:45:01.325711 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.325678    2569 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 03:45:01.325711 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.325685    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 03:45:01.325711 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.325698    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 03:45:01.326587 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.326578    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 03:45:01.329421 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.329410    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 03:45:01.329545 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.329521    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 03:45:01.329585 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.329564    2569 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 03:45:01.329585 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.329581    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 03:45:01.329653 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.329598    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 03:45:01.329653 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.329613    2569 kubelet.go:398] "Adding apiserver pod source"
Jul 30 03:45:01.329653 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.329625    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 03:45:01.329997 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.329985    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 03:45:01.330050 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.330002    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 03:45:01.333343 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.333328    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 03:45:01.335228 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.335214    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 03:45:01.335284 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.335244    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 03:45:01.336728 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.336715    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 03:45:01.336794 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.336731    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 03:45:01.336794 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.336737    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 03:45:01.336794 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.336743    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 03:45:01.336794 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.336751    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 03:45:01.336794 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.336759    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 03:45:01.336794 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.336770    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 03:45:01.336794 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.336776    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 03:45:01.336794 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.336782    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 03:45:01.336794 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.336787    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 03:45:01.336794 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.336795    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 03:45:01.336794 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.336801    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 03:45:01.337600 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.337578    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 03:45:01.337600 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.337598    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 03:45:01.339405 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.339382    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-104.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 03:45:01.339405 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.339391    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 03:45:01.339521 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.339446    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 03:45:01.340833 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.340819    2569 server.go:1268] "Started kubelet"
Jul 30 03:45:01.340961 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.340909    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 03:45:01.341015 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.341001    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 03:45:01.341230 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.341140    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 03:45:01.341953 ip-10-0-136-104 systemd[1]: Started Kubernetes Kubelet.
Jul 30 03:45:01.344444 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.344414    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 03:45:01.345299 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.345282    2569 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 03:45:01.348664 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.348642    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 03:45:01.353594 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.353579    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 03:45:01.353661 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.353621    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 03:45:01.354222 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.354205    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 03:45:01.354338 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.354205    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 03:45:01.354428 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.354414    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 03:45:01.354493 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.354274    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found"
Jul 30 03:45:01.354566 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.354550    2569 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 03:45:01.354566 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.354559    2569 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 03:45:01.354918 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.354901    2569 factory.go:55] Registering systemd factory
Jul 30 03:45:01.354969 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.354922    2569 factory.go:223] Registration of the systemd container factory successfully
Jul 30 03:45:01.355162 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.355148    2569 factory.go:153] Registering CRI-O factory
Jul 30 03:45:01.355217 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.355167    2569 factory.go:223] Registration of the crio container factory successfully
Jul 30 03:45:01.355217 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.355214    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 03:45:01.355309 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.355231    2569 factory.go:103] Registering Raw factory
Jul 30 03:45:01.355309 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.355245    2569 manager.go:1196] Started watching for new ooms in manager
Jul 30 03:45:01.355485 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.355463    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 30 03:45:01.355616 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.355593    2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 30 03:45:01.355885 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.355872    2569 manager.go:349] Starting recovery of all containers
Jul 30 03:45:01.356750 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.355508    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-104.ec2.internal.18c6f478a7672b84  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-104.ec2.internal,UID:ip-10-0-136-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-104.ec2.internal,},FirstTimestamp:2026-07-30 03:45:01.340789636 +0000 UTC m=+0.379500647,LastTimestamp:2026-07-30 03:45:01.340789636 +0000 UTC m=+0.379500647,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-104.ec2.internal,}"
Jul 30 03:45:01.366901 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.366752    2569 manager.go:354] Recovery completed
Jul 30 03:45:01.371158 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.371142    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 03:45:01.372010 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.371993    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hgvgt"
Jul 30 03:45:01.373880 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.373865    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 03:45:01.373939 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.373892    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 03:45:01.373939 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.373902    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientPID"
Jul 30 03:45:01.374359 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.374342    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 03:45:01.374359 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.374358    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 03:45:01.374473 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.374370    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 03:45:01.374473 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.374376    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 03:45:01.375116 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.375100    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 03:45:01.375186 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.375143    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 03:45:01.375186 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.375159    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientPID"
Jul 30 03:45:01.377154 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.377138    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hgvgt"
Jul 30 03:45:01.377225 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.377217    2569 policy_none.go:49] "None policy: Start"
Jul 30 03:45:01.377270 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.377230    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 03:45:01.377270 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.377241    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 03:45:01.380483 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.380471    2569 policy_none.go:47] "Start"
Jul 30 03:45:01.418122 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.418104    2569 manager.go:337] "Starting Device Plugin manager"
Jul 30 03:45:01.418243 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.418139    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 03:45:01.418243 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.418149    2569 server.go:87] "Starting device plugin registration server"
Jul 30 03:45:01.418631 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.418618    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 03:45:01.418682 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.418633    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 03:45:01.418768 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.418750    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 03:45:01.418842 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.418835    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 03:45:01.418894 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.418846    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 03:45:01.419410 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.419389    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 03:45:01.419485 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.419449    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-104.ec2.internal\" not found"
Jul 30 03:45:01.457504 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.457457    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 03:45:01.464127 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.458860    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 03:45:01.464127 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.458907    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 03:45:01.464127 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.458933    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 03:45:01.464127 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.458975    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 03:45:01.464127 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.461398    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 03:45:01.519525 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.519453    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 03:45:01.520432 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.520410    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 03:45:01.520554 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.520450    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 03:45:01.520554 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.520464    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientPID"
Jul 30 03:45:01.520554 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.520489    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.527762 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.527748    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.527811 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.527770    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-104.ec2.internal\": node \"ip-10-0-136-104.ec2.internal\" not found"
Jul 30 03:45:01.551151 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.551130    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found"
Jul 30 03:45:01.559679 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.559654    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal"]
Jul 30 03:45:01.559738 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.559720    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 03:45:01.560623 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.560602    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 03:45:01.560707 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.560638    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 03:45:01.560707 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.560649    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientPID"
Jul 30 03:45:01.562830 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.562818    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 03:45:01.562944 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.562928    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.562993 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.562958    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 03:45:01.563566 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.563547    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 03:45:01.563656 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.563580    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 03:45:01.563656 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.563593    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientPID"
Jul 30 03:45:01.563656 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.563553    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 03:45:01.563656 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.563657    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 03:45:01.563834 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.563666    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientPID"
Jul 30 03:45:01.565925 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.565909    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.566016 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.565939    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 03:45:01.566618 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.566601    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 03:45:01.566706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.566632    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 03:45:01.566706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.566646    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientPID"
Jul 30 03:45:01.592361 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.592338    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-104.ec2.internal\" not found" node="ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.596825 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.596800    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-104.ec2.internal\" not found" node="ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.651264 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.651226    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found"
Jul 30 03:45:01.655657 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.655642    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e735cf5928c25006e73eda857bc02787-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal\" (UID: \"e735cf5928c25006e73eda857bc02787\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.655713 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.655665    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e735cf5928c25006e73eda857bc02787-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal\" (UID: \"e735cf5928c25006e73eda857bc02787\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.655713 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.655687    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9ae53f74fb865d246d654758e9704219-config\") pod \"kube-apiserver-proxy-ip-10-0-136-104.ec2.internal\" (UID: \"9ae53f74fb865d246d654758e9704219\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.751758 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.751726    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found"
Jul 30 03:45:01.756002 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.755976    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e735cf5928c25006e73eda857bc02787-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal\" (UID: \"e735cf5928c25006e73eda857bc02787\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.756063 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.756011    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e735cf5928c25006e73eda857bc02787-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal\" (UID: \"e735cf5928c25006e73eda857bc02787\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.756063 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.756029    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9ae53f74fb865d246d654758e9704219-config\") pod \"kube-apiserver-proxy-ip-10-0-136-104.ec2.internal\" (UID: \"9ae53f74fb865d246d654758e9704219\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.756125 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.756078    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9ae53f74fb865d246d654758e9704219-config\") pod \"kube-apiserver-proxy-ip-10-0-136-104.ec2.internal\" (UID: \"9ae53f74fb865d246d654758e9704219\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.756125 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.756093    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e735cf5928c25006e73eda857bc02787-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal\" (UID: \"e735cf5928c25006e73eda857bc02787\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.756125 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.756093    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e735cf5928c25006e73eda857bc02787-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal\" (UID: \"e735cf5928c25006e73eda857bc02787\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.852509 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.852438    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found"
Jul 30 03:45:01.894944 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.894908    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.900668 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:01.900651    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:01.953561 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:01.953520    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found"
Jul 30 03:45:02.053991 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:02.053958    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found"
Jul 30 03:45:02.154457 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:02.154381    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found"
Jul 30 03:45:02.254849 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:02.254819    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found"
Jul 30 03:45:02.290099 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:02.290073    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 03:45:02.290504 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:02.290241    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 03:45:02.355006 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:02.354977    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found"
Jul 30 03:45:02.382276 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:02.382230    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 03:40:02 +0000 UTC" deadline="2028-03-06 07:41:02.28391251 +0000 UTC"
Jul 30 03:45:02.382276 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:02.382269    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14043h55m59.901646183s"
Jul 30 03:45:02.455533 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:02.455443    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found"
Jul 30 03:45:02.532498 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:02.532474    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 03:45:02.554435 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:02.554405    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:02.567826 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:02.567804    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 03:45:02.569828 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:02.569809    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal"
Jul 30 03:45:02.577264 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:02.577245    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 03:45:02.585867 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:02.585852    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 03:45:02.729336 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:02.729305    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode735cf5928c25006e73eda857bc02787.slice/crio-03c05222ff5b89ba7a8076563f47cd8b940fd2301445bacde7b81f47d1c715f3 WatchSource:0}: Error finding container 03c05222ff5b89ba7a8076563f47cd8b940fd2301445bacde7b81f47d1c715f3: Status 404 returned error can't find the container with id 03c05222ff5b89ba7a8076563f47cd8b940fd2301445bacde7b81f47d1c715f3
Jul 30 03:45:02.729782 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:02.729757    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ae53f74fb865d246d654758e9704219.slice/crio-d997187212b47c0a3977d5689243f7717d9ff010f1df394ad35bd5de39cfba2e WatchSource:0}: Error finding container d997187212b47c0a3977d5689243f7717d9ff010f1df394ad35bd5de39cfba2e: Status 404 returned error can't find the container with id d997187212b47c0a3977d5689243f7717d9ff010f1df394ad35bd5de39cfba2e
Jul 30 03:45:02.733760 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:02.733740    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 03:45:02.755248 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:02.755230    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 03:45:03.332461 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.332421    2569 apiserver.go:52] "Watching apiserver"
Jul 30 03:45:03.337988 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.337972    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 03:45:03.339838 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.339818    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-6887b","kube-system/global-pull-secret-syncer-vxddq","kube-system/konnectivity-agent-fxtxs","kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal","openshift-dns/node-resolver-zdwwk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal","openshift-multus/multus-d6x52","openshift-multus/network-metrics-daemon-nsf2h","openshift-network-diagnostics/network-check-target-bpwlj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz","openshift-cluster-node-tuning-operator/tuned-sll89","openshift-image-registry/node-ca-5dzsm","openshift-multus/multus-additional-cni-plugins-mbpx8","openshift-network-operator/iptables-alerter-z8gsc"]
Jul 30 03:45:03.343861 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.343837    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.346173 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.346151    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 03:45:03.346306 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.346173    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 03:45:03.346389 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.346365    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.347505 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.346871    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 03:45:03.347505 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.346808    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-drcks\""
Jul 30 03:45:03.350987 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.349484    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fxtxs"
Jul 30 03:45:03.350987 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.349553    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 03:45:03.350987 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.349762    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 03:45:03.350987 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.350143    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 03:45:03.350987 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.350338    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 03:45:03.350987 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.350473    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dn44q\""
Jul 30 03:45:03.350987 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.350863    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 03:45:03.352407 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.352385    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 03:45:03.352712 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.352664    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z8gsc"
Jul 30 03:45:03.352712 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.352690    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 03:45:03.352926 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.352906    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hvhkn\""
Jul 30 03:45:03.354461 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.354426    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 03:45:03.354695 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.354673    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 03:45:03.354797 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.354705    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 03:45:03.354797 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.354674    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l4p6c\""
Jul 30 03:45:03.354911 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.354819    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zdwwk"
Jul 30 03:45:03.355137 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.355120    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 03:45:03.356894 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.356876    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 03:45:03.356988 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.356879    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 03:45:03.356988 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.356940    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wcz7j\""
Jul 30 03:45:03.357261 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.357246    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:03.357343 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:03.357320    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:03.359603 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.359531    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:03.359701 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:03.359656    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:03.362044 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.362026    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.363848 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.363816    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 03:45:03.363848 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.363830    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 03:45:03.364001 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.363898    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fr24d\""
Jul 30 03:45:03.364554 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.364531    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:03.364635 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:03.364602    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:03.365312 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365124    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db777952-f0cd-4473-8c10-5ad3c4d1af11-tmp-dir\") pod \"node-resolver-zdwwk\" (UID: \"db777952-f0cd-4473-8c10-5ad3c4d1af11\") " pod="openshift-dns/node-resolver-zdwwk"
Jul 30 03:45:03.365312 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365154    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-socket-dir\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.365312 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365179    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-etc-selinux\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.365312 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365205    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6583ab82-829c-4f0e-829f-0c7ccc835aa4-iptables-alerter-script\") pod \"iptables-alerter-z8gsc\" (UID: \"6583ab82-829c-4f0e-829f-0c7ccc835aa4\") " pod="openshift-network-operator/iptables-alerter-z8gsc"
Jul 30 03:45:03.365312 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365237    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7h7w\" (UniqueName: \"kubernetes.io/projected/db777952-f0cd-4473-8c10-5ad3c4d1af11-kube-api-access-w7h7w\") pod \"node-resolver-zdwwk\" (UID: \"db777952-f0cd-4473-8c10-5ad3c4d1af11\") " pod="openshift-dns/node-resolver-zdwwk"
Jul 30 03:45:03.365312 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365262    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-cnibin\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.365312 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365285    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-os-release\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.365312 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365307    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-cni-binary-copy\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.365740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365331    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/80d2e732-5a45-4f32-bec1-768138b5d8ae-konnectivity-ca\") pod \"konnectivity-agent-fxtxs\" (UID: \"80d2e732-5a45-4f32-bec1-768138b5d8ae\") " pod="kube-system/konnectivity-agent-fxtxs"
Jul 30 03:45:03.365740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365356    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.365740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365397    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-device-dir\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.365740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365425    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-system-cni-dir\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.365740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365453    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.365740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365517    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94kdx\" (UniqueName: \"kubernetes.io/projected/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-kube-api-access-94kdx\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.365740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365548    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/80d2e732-5a45-4f32-bec1-768138b5d8ae-agent-certs\") pod \"konnectivity-agent-fxtxs\" (UID: \"80d2e732-5a45-4f32-bec1-768138b5d8ae\") " pod="kube-system/konnectivity-agent-fxtxs"
Jul 30 03:45:03.365740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365623    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xftzs\" (UniqueName: \"kubernetes.io/projected/6583ab82-829c-4f0e-829f-0c7ccc835aa4-kube-api-access-xftzs\") pod \"iptables-alerter-z8gsc\" (UID: \"6583ab82-829c-4f0e-829f-0c7ccc835aa4\") " pod="openshift-network-operator/iptables-alerter-z8gsc"
Jul 30 03:45:03.365740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365677    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db777952-f0cd-4473-8c10-5ad3c4d1af11-hosts-file\") pod \"node-resolver-zdwwk\" (UID: \"db777952-f0cd-4473-8c10-5ad3c4d1af11\") " pod="openshift-dns/node-resolver-zdwwk"
Jul 30 03:45:03.365740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365704    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-registration-dir\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.366160 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365736    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-sys-fs\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.366160 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365765    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfvw5\" (UniqueName: \"kubernetes.io/projected/3e669450-29a8-429b-abeb-31f57c441dda-kube-api-access-qfvw5\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.366160 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365780    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.366160 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365812    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.366160 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.365847    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6583ab82-829c-4f0e-829f-0c7ccc835aa4-host-slash\") pod \"iptables-alerter-z8gsc\" (UID: \"6583ab82-829c-4f0e-829f-0c7ccc835aa4\") " pod="openshift-network-operator/iptables-alerter-z8gsc"
Jul 30 03:45:03.367190 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.367173    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.369143 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.369127    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 03:45:03.369726 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.369549    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 03:45:03.369726 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.369619    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 03:45:03.369726 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.369633    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.369954 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.369881    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 03:45:03.369954 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.369924    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 03:45:03.369954 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.369946    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 03:45:03.370106 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.370016    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-n7gjf\""
Jul 30 03:45:03.371886 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.371824    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 03:45:03.371886 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.371829    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 03:45:03.372137 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.372120    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-87lxp\""
Jul 30 03:45:03.372586 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.372544    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5dzsm"
Jul 30 03:45:03.374426 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.374389    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 03:45:03.374708 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.374690    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 03:45:03.374805 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.374723    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 03:45:03.374868 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.374815    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rrb25\""
Jul 30 03:45:03.390787 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.390766    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9tbhl"
Jul 30 03:45:03.398593 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.398577    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9tbhl"
Jul 30 03:45:03.455727 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.455705    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 03:45:03.463374 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.463340    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal" event={"ID":"9ae53f74fb865d246d654758e9704219","Type":"ContainerStarted","Data":"d997187212b47c0a3977d5689243f7717d9ff010f1df394ad35bd5de39cfba2e"}
Jul 30 03:45:03.464523 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.464502    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" event={"ID":"e735cf5928c25006e73eda857bc02787","Type":"ContainerStarted","Data":"03c05222ff5b89ba7a8076563f47cd8b940fd2301445bacde7b81f47d1c715f3"}
Jul 30 03:45:03.466040 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466023    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:03.466141 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466054    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-sysctl-d\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.466141 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466080    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-slash\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.466141 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466103    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-multus-conf-dir\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.466141 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466125    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-etc-kubernetes\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.466324 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466163    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-modprobe-d\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.466324 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466205    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-systemd-units\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.466324 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466237    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/79c63a29-b6fa-4412-9446-79a29e51deef-env-overrides\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.466324 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466267    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/79c63a29-b6fa-4412-9446-79a29e51deef-ovn-node-metrics-cert\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.466324 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466299    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6583ab82-829c-4f0e-829f-0c7ccc835aa4-host-slash\") pod \"iptables-alerter-z8gsc\" (UID: \"6583ab82-829c-4f0e-829f-0c7ccc835aa4\") " pod="openshift-network-operator/iptables-alerter-z8gsc"
Jul 30 03:45:03.466565 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466331    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7h7w\" (UniqueName: \"kubernetes.io/projected/db777952-f0cd-4473-8c10-5ad3c4d1af11-kube-api-access-w7h7w\") pod \"node-resolver-zdwwk\" (UID: \"db777952-f0cd-4473-8c10-5ad3c4d1af11\") " pod="openshift-dns/node-resolver-zdwwk"
Jul 30 03:45:03.466565 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466353    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6583ab82-829c-4f0e-829f-0c7ccc835aa4-host-slash\") pod \"iptables-alerter-z8gsc\" (UID: \"6583ab82-829c-4f0e-829f-0c7ccc835aa4\") " pod="openshift-network-operator/iptables-alerter-z8gsc"
Jul 30 03:45:03.466565 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-node-log\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.466565 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466392    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-multus-socket-dir-parent\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.466565 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466419    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-hostroot\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.466565 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466444    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a26002ce-a680-4af4-9513-044a00d2393d-kubelet-config\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:03.466565 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466472    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-socket-dir\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.466565 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466496    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-run-systemd\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.466565 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466521    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-run-k8s-cni-cncf-io\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.466565 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466563    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-run-netns\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.466565 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466579    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-var-lib-kubelet\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.466565 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466579    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-socket-dir\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466605    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-run-netns\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466637    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dd18769b-ddef-4460-980b-528c13aa0040-etc-tuned\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466659    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjzsr\" (UniqueName: \"kubernetes.io/projected/79c63a29-b6fa-4412-9446-79a29e51deef-kube-api-access-sjzsr\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-var-lib-cni-multus\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466711    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-device-dir\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466744    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-system-cni-dir\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466783    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-system-cni-dir\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466808    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-device-dir\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466813    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94kdx\" (UniqueName: \"kubernetes.io/projected/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-kube-api-access-94kdx\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466835    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-kubernetes\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466851    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b900747-00b2-49fe-8af7-ab67f091664c-serviceca\") pod \"node-ca-5dzsm\" (UID: \"4b900747-00b2-49fe-8af7-ab67f091664c\") " pod="openshift-image-registry/node-ca-5dzsm"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466876    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs\") pod \"network-metrics-daemon-nsf2h\" (UID: \"30c27db8-8cfe-492d-855d-4e085771ed3e\") " pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.466923    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nc4g\" (UniqueName: \"kubernetes.io/projected/30c27db8-8cfe-492d-855d-4e085771ed3e-kube-api-access-4nc4g\") pod \"network-metrics-daemon-nsf2h\" (UID: \"30c27db8-8cfe-492d-855d-4e085771ed3e\") " pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467036    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-registration-dir\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.467085 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467071    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467130    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-registration-dir\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467136    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-var-lib-kubelet\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467172    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-var-lib-openvswitch\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467202    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-run-openvswitch\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467228    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-systemd\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467230    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467261    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-sys\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467294    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-cnibin\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-run-ovn\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467333    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-cni-netd\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467364    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-cnibin\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467375    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b178858-acff-4796-87a6-9852811da3c6-multus-daemon-config\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467416    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-run-multus-certs\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467446    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a26002ce-a680-4af4-9513-044a00d2393d-dbus\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467475    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db777952-f0cd-4473-8c10-5ad3c4d1af11-tmp-dir\") pod \"node-resolver-zdwwk\" (UID: \"db777952-f0cd-4473-8c10-5ad3c4d1af11\") " pod="openshift-dns/node-resolver-zdwwk"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467491    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-sysconfig\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.467998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467508    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467526    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db777952-f0cd-4473-8c10-5ad3c4d1af11-hosts-file\") pod \"node-resolver-zdwwk\" (UID: \"db777952-f0cd-4473-8c10-5ad3c4d1af11\") " pod="openshift-dns/node-resolver-zdwwk"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467545    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sc6g\" (UniqueName: \"kubernetes.io/projected/4b900747-00b2-49fe-8af7-ab67f091664c-kube-api-access-7sc6g\") pod \"node-ca-5dzsm\" (UID: \"4b900747-00b2-49fe-8af7-ab67f091664c\") " pod="openshift-image-registry/node-ca-5dzsm"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467591    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-var-lib-cni-bin\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467615    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db777952-f0cd-4473-8c10-5ad3c4d1af11-hosts-file\") pod \"node-resolver-zdwwk\" (UID: \"db777952-f0cd-4473-8c10-5ad3c4d1af11\") " pod="openshift-dns/node-resolver-zdwwk"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467682    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-cni-bin\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467706    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfvw5\" (UniqueName: \"kubernetes.io/projected/3e669450-29a8-429b-abeb-31f57c441dda-kube-api-access-qfvw5\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467723    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467762    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-run\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467778    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-log-socket\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467794    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b178858-acff-4796-87a6-9852811da3c6-cni-binary-copy\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467810    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtfmp\" (UniqueName: \"kubernetes.io/projected/1b178858-acff-4796-87a6-9852811da3c6-kube-api-access-rtfmp\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467838    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-etc-selinux\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467864    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6583ab82-829c-4f0e-829f-0c7ccc835aa4-iptables-alerter-script\") pod \"iptables-alerter-z8gsc\" (UID: \"6583ab82-829c-4f0e-829f-0c7ccc835aa4\") " pod="openshift-network-operator/iptables-alerter-z8gsc"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467888    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-host\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467927    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-etc-openvswitch\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467953    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-run-ovn-kubernetes\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.468798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.467988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b900747-00b2-49fe-8af7-ab67f091664c-host\") pod \"node-ca-5dzsm\" (UID: \"4b900747-00b2-49fe-8af7-ab67f091664c\") " pod="openshift-image-registry/node-ca-5dzsm"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468012    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-multus-cni-dir\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468038    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-kubelet\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468041    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-etc-selinux\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468069    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468040    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db777952-f0cd-4473-8c10-5ad3c4d1af11-tmp-dir\") pod \"node-resolver-zdwwk\" (UID: \"db777952-f0cd-4473-8c10-5ad3c4d1af11\") " pod="openshift-dns/node-resolver-zdwwk"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468095    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-os-release\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468112    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-cni-binary-copy\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468133    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/80d2e732-5a45-4f32-bec1-768138b5d8ae-konnectivity-ca\") pod \"konnectivity-agent-fxtxs\" (UID: \"80d2e732-5a45-4f32-bec1-768138b5d8ae\") " pod="kube-system/konnectivity-agent-fxtxs"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468158    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-sysctl-conf\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468195    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-os-release\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468201    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-lib-modules\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468247    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd18769b-ddef-4460-980b-528c13aa0040-tmp\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468273    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468296    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/80d2e732-5a45-4f32-bec1-768138b5d8ae-agent-certs\") pod \"konnectivity-agent-fxtxs\" (UID: \"80d2e732-5a45-4f32-bec1-768138b5d8ae\") " pod="kube-system/konnectivity-agent-fxtxs"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xftzs\" (UniqueName: \"kubernetes.io/projected/6583ab82-829c-4f0e-829f-0c7ccc835aa4-kube-api-access-xftzs\") pod \"iptables-alerter-z8gsc\" (UID: \"6583ab82-829c-4f0e-829f-0c7ccc835aa4\") " pod="openshift-network-operator/iptables-alerter-z8gsc"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468325    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.469365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468357    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slhq9\" (UniqueName: \"kubernetes.io/projected/dd18769b-ddef-4460-980b-528c13aa0040-kube-api-access-slhq9\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.469953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468364    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.469953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468342    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.469953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468388    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-system-cni-dir\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.469953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468417    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-sys-fs\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.469953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468470    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/79c63a29-b6fa-4412-9446-79a29e51deef-ovnkube-config\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.469953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468504    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6583ab82-829c-4f0e-829f-0c7ccc835aa4-iptables-alerter-script\") pod \"iptables-alerter-z8gsc\" (UID: \"6583ab82-829c-4f0e-829f-0c7ccc835aa4\") " pod="openshift-network-operator/iptables-alerter-z8gsc"
Jul 30 03:45:03.469953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468502    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/79c63a29-b6fa-4412-9446-79a29e51deef-ovnkube-script-lib\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.469953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468557    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cmgt\" (UniqueName: \"kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt\") pod \"network-check-target-bpwlj\" (UID: \"0295adc4-b5c7-42c2-ae4c-b653ae27ca5c\") " pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:03.469953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468587    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-cnibin\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.469953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468597    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e669450-29a8-429b-abeb-31f57c441dda-sys-fs\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.469953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468643    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-os-release\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.469953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468715    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-cni-binary-copy\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.469953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468828    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/80d2e732-5a45-4f32-bec1-768138b5d8ae-konnectivity-ca\") pod \"konnectivity-agent-fxtxs\" (UID: \"80d2e732-5a45-4f32-bec1-768138b5d8ae\") " pod="kube-system/konnectivity-agent-fxtxs"
Jul 30 03:45:03.469953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.468828    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 03:45:03.471979 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.471961    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/80d2e732-5a45-4f32-bec1-768138b5d8ae-agent-certs\") pod \"konnectivity-agent-fxtxs\" (UID: \"80d2e732-5a45-4f32-bec1-768138b5d8ae\") " pod="kube-system/konnectivity-agent-fxtxs"
Jul 30 03:45:03.474126 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.474102    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7h7w\" (UniqueName: \"kubernetes.io/projected/db777952-f0cd-4473-8c10-5ad3c4d1af11-kube-api-access-w7h7w\") pod \"node-resolver-zdwwk\" (UID: \"db777952-f0cd-4473-8c10-5ad3c4d1af11\") " pod="openshift-dns/node-resolver-zdwwk"
Jul 30 03:45:03.474717 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.474695    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94kdx\" (UniqueName: \"kubernetes.io/projected/621e8000-b5ef-4835-ad07-5fa5cd6e34e5-kube-api-access-94kdx\") pod \"multus-additional-cni-plugins-mbpx8\" (UID: \"621e8000-b5ef-4835-ad07-5fa5cd6e34e5\") " pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.474971 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.474954    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfvw5\" (UniqueName: \"kubernetes.io/projected/3e669450-29a8-429b-abeb-31f57c441dda-kube-api-access-qfvw5\") pod \"aws-ebs-csi-driver-node-gftvz\" (UID: \"3e669450-29a8-429b-abeb-31f57c441dda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.475392 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.475372    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xftzs\" (UniqueName: \"kubernetes.io/projected/6583ab82-829c-4f0e-829f-0c7ccc835aa4-kube-api-access-xftzs\") pod \"iptables-alerter-z8gsc\" (UID: \"6583ab82-829c-4f0e-829f-0c7ccc835aa4\") " pod="openshift-network-operator/iptables-alerter-z8gsc"
Jul 30 03:45:03.568933 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.568902    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-multus-cni-dir\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.568933 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.568933    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-kubelet\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.569127 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.568959    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.569127 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.568991    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-sysctl-conf\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.569127 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569015    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-lib-modules\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.569127 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569038    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd18769b-ddef-4460-980b-528c13aa0040-tmp\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.569127 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569063    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slhq9\" (UniqueName: \"kubernetes.io/projected/dd18769b-ddef-4460-980b-528c13aa0040-kube-api-access-slhq9\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.569127 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569089    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-system-cni-dir\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.569127 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569121    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/79c63a29-b6fa-4412-9446-79a29e51deef-ovnkube-config\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.569127 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569121    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-multus-cni-dir\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.569551 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569146    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/79c63a29-b6fa-4412-9446-79a29e51deef-ovnkube-script-lib\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.569551 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569171    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cmgt\" (UniqueName: \"kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt\") pod \"network-check-target-bpwlj\" (UID: \"0295adc4-b5c7-42c2-ae4c-b653ae27ca5c\") " pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:03.569551 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569198    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-cnibin\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.569551 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569213    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-os-release\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.569551 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569242    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:03.569551 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569257    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-sysctl-d\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.569551 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569265    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-lib-modules\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.569551 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569308    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-slash\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.569551 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569348    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-kubelet\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.569551 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569384    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.569551 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569493    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-sysctl-conf\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.569551 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569272    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-slash\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.569551 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569537    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-multus-conf-dir\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569565    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-etc-kubernetes\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569609    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-modprobe-d\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569635    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-systemd-units\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569658    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/79c63a29-b6fa-4412-9446-79a29e51deef-env-overrides\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569684    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/79c63a29-b6fa-4412-9446-79a29e51deef-ovn-node-metrics-cert\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569714    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-node-log\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569741    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-multus-socket-dir-parent\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569765    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-hostroot\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569791    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a26002ce-a680-4af4-9513-044a00d2393d-kubelet-config\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569818    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-run-systemd\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569844    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-run-k8s-cni-cncf-io\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569849    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-system-cni-dir\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569869    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-run-netns\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569895    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-var-lib-kubelet\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569920    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-run-netns\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569943    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dd18769b-ddef-4460-980b-528c13aa0040-etc-tuned\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569967    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjzsr\" (UniqueName: \"kubernetes.io/projected/79c63a29-b6fa-4412-9446-79a29e51deef-kube-api-access-sjzsr\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.570263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.569992    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-var-lib-cni-multus\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570021    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-kubernetes\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570052    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b900747-00b2-49fe-8af7-ab67f091664c-serviceca\") pod \"node-ca-5dzsm\" (UID: \"4b900747-00b2-49fe-8af7-ab67f091664c\") " pod="openshift-image-registry/node-ca-5dzsm"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570077    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs\") pod \"network-metrics-daemon-nsf2h\" (UID: \"30c27db8-8cfe-492d-855d-4e085771ed3e\") " pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570093    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nc4g\" (UniqueName: \"kubernetes.io/projected/30c27db8-8cfe-492d-855d-4e085771ed3e-kube-api-access-4nc4g\") pod \"network-metrics-daemon-nsf2h\" (UID: \"30c27db8-8cfe-492d-855d-4e085771ed3e\") " pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570111    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-var-lib-kubelet\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570136    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-var-lib-openvswitch\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570153    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-run-openvswitch\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570172    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-systemd\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570194    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-sys\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570221    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-run-ovn\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570251    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-cni-netd\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570275    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b178858-acff-4796-87a6-9852811da3c6-multus-daemon-config\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570299    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-run-multus-certs\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570320    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a26002ce-a680-4af4-9513-044a00d2393d-dbus\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570336    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-sysconfig\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570354    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sc6g\" (UniqueName: \"kubernetes.io/projected/4b900747-00b2-49fe-8af7-ab67f091664c-kube-api-access-7sc6g\") pod \"node-ca-5dzsm\" (UID: \"4b900747-00b2-49fe-8af7-ab67f091664c\") " pod="openshift-image-registry/node-ca-5dzsm"
Jul 30 03:45:03.571165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570354    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/79c63a29-b6fa-4412-9446-79a29e51deef-ovnkube-config\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570370    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-var-lib-cni-bin\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570395    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-cni-bin\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570422    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-run\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-log-socket\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570475    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b178858-acff-4796-87a6-9852811da3c6-cni-binary-copy\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570501    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtfmp\" (UniqueName: \"kubernetes.io/projected/1b178858-acff-4796-87a6-9852811da3c6-kube-api-access-rtfmp\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570540    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-host\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570567    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-etc-openvswitch\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570595    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-run-ovn-kubernetes\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570638    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b900747-00b2-49fe-8af7-ab67f091664c-host\") pod \"node-ca-5dzsm\" (UID: \"4b900747-00b2-49fe-8af7-ab67f091664c\") " pod="openshift-image-registry/node-ca-5dzsm"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570729    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b900747-00b2-49fe-8af7-ab67f091664c-host\") pod \"node-ca-5dzsm\" (UID: \"4b900747-00b2-49fe-8af7-ab67f091664c\") " pod="openshift-image-registry/node-ca-5dzsm"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570769    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-multus-conf-dir\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570793    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-etc-kubernetes\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570825    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/79c63a29-b6fa-4412-9446-79a29e51deef-ovnkube-script-lib\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570872    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-modprobe-d\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570896    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-var-lib-kubelet\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570916    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-var-lib-openvswitch\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.572080 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570929    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-systemd-units\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.570961    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-run-openvswitch\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571297    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/79c63a29-b6fa-4412-9446-79a29e51deef-env-overrides\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571364    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-systemd\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571416    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-sys\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571458    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-run-ovn\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571499    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-cnibin\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571503    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-cni-netd\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571548    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-run-multus-certs\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:03.571618    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571625    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-os-release\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571657    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-node-log\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:03.571691    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret podName:a26002ce-a680-4af4-9513-044a00d2393d nodeName:}" failed. No retries permitted until 2026-07-30 03:45:04.07165801 +0000 UTC m=+3.110369020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret") pod "global-pull-secret-syncer-vxddq" (UID: "a26002ce-a680-4af4-9513-044a00d2393d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571730    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-multus-socket-dir-parent\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571741    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a26002ce-a680-4af4-9513-044a00d2393d-dbus\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571778    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-sysctl-d\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571793    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a26002ce-a680-4af4-9513-044a00d2393d-kubelet-config\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571799    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-sysconfig\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.572962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571840    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-hostroot\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571848    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-run-k8s-cni-cncf-io\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571893    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-run-netns\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571936    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-var-lib-kubelet\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571980    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-run-netns\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.571989    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-var-lib-cni-bin\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.572050    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-host\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.572051    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-cni-bin\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.572095    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-run\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.572102    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-etc-openvswitch\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.572141    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-log-socket\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.572145    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-host-run-ovn-kubernetes\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.572362    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/79c63a29-b6fa-4412-9446-79a29e51deef-run-systemd\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.572644    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd18769b-ddef-4460-980b-528c13aa0040-etc-kubernetes\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.572678    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b178858-acff-4796-87a6-9852811da3c6-cni-binary-copy\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.572737    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b178858-acff-4796-87a6-9852811da3c6-host-var-lib-cni-multus\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:03.572823    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:03.572895    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs podName:30c27db8-8cfe-492d-855d-4e085771ed3e nodeName:}" failed. No retries permitted until 2026-07-30 03:45:04.072874921 +0000 UTC m=+3.111585941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs") pod "network-metrics-daemon-nsf2h" (UID: "30c27db8-8cfe-492d-855d-4e085771ed3e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:45:03.573774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.573266    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b900747-00b2-49fe-8af7-ab67f091664c-serviceca\") pod \"node-ca-5dzsm\" (UID: \"4b900747-00b2-49fe-8af7-ab67f091664c\") " pod="openshift-image-registry/node-ca-5dzsm"
Jul 30 03:45:03.574425 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.573351    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b178858-acff-4796-87a6-9852811da3c6-multus-daemon-config\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.574425 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.574397    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd18769b-ddef-4460-980b-528c13aa0040-tmp\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.574577 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.574531    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dd18769b-ddef-4460-980b-528c13aa0040-etc-tuned\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.574667 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:03.574618    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 03:45:03.574667 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:03.574642    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 03:45:03.574667 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:03.574656    2569 projected.go:196] Error preparing data for projected volume kube-api-access-5cmgt for pod openshift-network-diagnostics/network-check-target-bpwlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:45:03.574809 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:03.574766    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt podName:0295adc4-b5c7-42c2-ae4c-b653ae27ca5c nodeName:}" failed. No retries permitted until 2026-07-30 03:45:04.074747002 +0000 UTC m=+3.113458025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5cmgt" (UniqueName: "kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt") pod "network-check-target-bpwlj" (UID: "0295adc4-b5c7-42c2-ae4c-b653ae27ca5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:45:03.575289 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.575270    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/79c63a29-b6fa-4412-9446-79a29e51deef-ovn-node-metrics-cert\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.583301 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.583240    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slhq9\" (UniqueName: \"kubernetes.io/projected/dd18769b-ddef-4460-980b-528c13aa0040-kube-api-access-slhq9\") pod \"tuned-sll89\" (UID: \"dd18769b-ddef-4460-980b-528c13aa0040\") " pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.584155 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.583882    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nc4g\" (UniqueName: \"kubernetes.io/projected/30c27db8-8cfe-492d-855d-4e085771ed3e-kube-api-access-4nc4g\") pod \"network-metrics-daemon-nsf2h\" (UID: \"30c27db8-8cfe-492d-855d-4e085771ed3e\") " pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:03.584155 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.584114    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtfmp\" (UniqueName: \"kubernetes.io/projected/1b178858-acff-4796-87a6-9852811da3c6-kube-api-access-rtfmp\") pod \"multus-d6x52\" (UID: \"1b178858-acff-4796-87a6-9852811da3c6\") " pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.584924 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.584902    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjzsr\" (UniqueName: \"kubernetes.io/projected/79c63a29-b6fa-4412-9446-79a29e51deef-kube-api-access-sjzsr\") pod \"ovnkube-node-6887b\" (UID: \"79c63a29-b6fa-4412-9446-79a29e51deef\") " pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.585604 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.585583    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sc6g\" (UniqueName: \"kubernetes.io/projected/4b900747-00b2-49fe-8af7-ab67f091664c-kube-api-access-7sc6g\") pod \"node-ca-5dzsm\" (UID: \"4b900747-00b2-49fe-8af7-ab67f091664c\") " pod="openshift-image-registry/node-ca-5dzsm"
Jul 30 03:45:03.656319 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.656296    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz"
Jul 30 03:45:03.661789 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.661768    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mbpx8"
Jul 30 03:45:03.668118 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:03.668088    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e669450_29a8_429b_abeb_31f57c441dda.slice/crio-826b67c7078a1bdf699680a523f6a7f17d894bf9519de7ba3572cab5368278b5 WatchSource:0}: Error finding container 826b67c7078a1bdf699680a523f6a7f17d894bf9519de7ba3572cab5368278b5: Status 404 returned error can't find the container with id 826b67c7078a1bdf699680a523f6a7f17d894bf9519de7ba3572cab5368278b5
Jul 30 03:45:03.669310 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.669289    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fxtxs"
Jul 30 03:45:03.673192 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:03.672791    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621e8000_b5ef_4835_ad07_5fa5cd6e34e5.slice/crio-2a7a80e6008ccd417d2a427e14c51fce37f3fd40123858897945615aa3800bbe WatchSource:0}: Error finding container 2a7a80e6008ccd417d2a427e14c51fce37f3fd40123858897945615aa3800bbe: Status 404 returned error can't find the container with id 2a7a80e6008ccd417d2a427e14c51fce37f3fd40123858897945615aa3800bbe
Jul 30 03:45:03.675513 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.675481    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z8gsc"
Jul 30 03:45:03.679032 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:03.679001    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80d2e732_5a45_4f32_bec1_768138b5d8ae.slice/crio-16f2052973b81990e6a76c0880181290ec661c0c3fdedbbfb1139706a30b8f45 WatchSource:0}: Error finding container 16f2052973b81990e6a76c0880181290ec661c0c3fdedbbfb1139706a30b8f45: Status 404 returned error can't find the container with id 16f2052973b81990e6a76c0880181290ec661c0c3fdedbbfb1139706a30b8f45
Jul 30 03:45:03.681470 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.681451    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zdwwk"
Jul 30 03:45:03.685835 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:03.685808    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6583ab82_829c_4f0e_829f_0c7ccc835aa4.slice/crio-8dffe05aa67cb21d5d181d4454f13ddf025e3d53b4d43115db70dcb63a7ac917 WatchSource:0}: Error finding container 8dffe05aa67cb21d5d181d4454f13ddf025e3d53b4d43115db70dcb63a7ac917: Status 404 returned error can't find the container with id 8dffe05aa67cb21d5d181d4454f13ddf025e3d53b4d43115db70dcb63a7ac917
Jul 30 03:45:03.687151 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.687008    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d6x52"
Jul 30 03:45:03.693258 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.693145    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:03.693714 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:03.693686    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb777952_f0cd_4473_8c10_5ad3c4d1af11.slice/crio-88b83d3be90fbbbec032d77b775eb7b565a75aeaaf8dfa2be35e56a6bd378d37 WatchSource:0}: Error finding container 88b83d3be90fbbbec032d77b775eb7b565a75aeaaf8dfa2be35e56a6bd378d37: Status 404 returned error can't find the container with id 88b83d3be90fbbbec032d77b775eb7b565a75aeaaf8dfa2be35e56a6bd378d37
Jul 30 03:45:03.698623 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.698599    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sll89"
Jul 30 03:45:03.700956 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:03.700936    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b178858_acff_4796_87a6_9852811da3c6.slice/crio-9b3e76d6c9effd1f6dc09670049d025aefa59b657e681649104a727eaf4c4ca8 WatchSource:0}: Error finding container 9b3e76d6c9effd1f6dc09670049d025aefa59b657e681649104a727eaf4c4ca8: Status 404 returned error can't find the container with id 9b3e76d6c9effd1f6dc09670049d025aefa59b657e681649104a727eaf4c4ca8
Jul 30 03:45:03.705026 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:03.705002    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c63a29_b6fa_4412_9446_79a29e51deef.slice/crio-1ee50abe7c8cee4e4af82ca035daa4caf093a1f892594e57eb5a02c1ce640c30 WatchSource:0}: Error finding container 1ee50abe7c8cee4e4af82ca035daa4caf093a1f892594e57eb5a02c1ce640c30: Status 404 returned error can't find the container with id 1ee50abe7c8cee4e4af82ca035daa4caf093a1f892594e57eb5a02c1ce640c30
Jul 30 03:45:03.705330 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.705309    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5dzsm"
Jul 30 03:45:03.708650 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:03.708601    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd18769b_ddef_4460_980b_528c13aa0040.slice/crio-e5a452a1e72a2158afe7f6eb8452ce964d753fdb648a07b27a8b188fdf5ac9fc WatchSource:0}: Error finding container e5a452a1e72a2158afe7f6eb8452ce964d753fdb648a07b27a8b188fdf5ac9fc: Status 404 returned error can't find the container with id e5a452a1e72a2158afe7f6eb8452ce964d753fdb648a07b27a8b188fdf5ac9fc
Jul 30 03:45:03.716303 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:03.716261    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b900747_00b2_49fe_8af7_ab67f091664c.slice/crio-e4a6ca140f60808210af41bf62034b14294c3b14a349b192646d67fa2057edc3 WatchSource:0}: Error finding container e4a6ca140f60808210af41bf62034b14294c3b14a349b192646d67fa2057edc3: Status 404 returned error can't find the container with id e4a6ca140f60808210af41bf62034b14294c3b14a349b192646d67fa2057edc3
Jul 30 03:45:03.772159 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:03.772130    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 03:45:04.074330 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.073658    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:04.074330 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.073723    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs\") pod \"network-metrics-daemon-nsf2h\" (UID: \"30c27db8-8cfe-492d-855d-4e085771ed3e\") " pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:04.074330 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:04.073863    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:45:04.074330 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:04.073926    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs podName:30c27db8-8cfe-492d-855d-4e085771ed3e nodeName:}" failed. No retries permitted until 2026-07-30 03:45:05.073907066 +0000 UTC m=+4.112618068 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs") pod "network-metrics-daemon-nsf2h" (UID: "30c27db8-8cfe-492d-855d-4e085771ed3e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:45:04.074330 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:04.074248    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 03:45:04.074330 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:04.074297    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret podName:a26002ce-a680-4af4-9513-044a00d2393d nodeName:}" failed. No retries permitted until 2026-07-30 03:45:05.074282132 +0000 UTC m=+4.112993133 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret") pod "global-pull-secret-syncer-vxddq" (UID: "a26002ce-a680-4af4-9513-044a00d2393d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 03:45:04.174992 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.174836    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cmgt\" (UniqueName: \"kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt\") pod \"network-check-target-bpwlj\" (UID: \"0295adc4-b5c7-42c2-ae4c-b653ae27ca5c\") " pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:04.175155 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:04.175060    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 03:45:04.175155 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:04.175087    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 03:45:04.175155 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:04.175103    2569 projected.go:196] Error preparing data for projected volume kube-api-access-5cmgt for pod openshift-network-diagnostics/network-check-target-bpwlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:45:04.175321 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:04.175168    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt podName:0295adc4-b5c7-42c2-ae4c-b653ae27ca5c nodeName:}" failed. No retries permitted until 2026-07-30 03:45:05.175148237 +0000 UTC m=+4.213859241 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5cmgt" (UniqueName: "kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt") pod "network-check-target-bpwlj" (UID: "0295adc4-b5c7-42c2-ae4c-b653ae27ca5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:45:04.401730 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.401680    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 03:40:04 +0000 UTC" deadline="2028-02-09 03:43:59.173785942 +0000 UTC"
Jul 30 03:45:04.401730 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.401725    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13415h58m54.772063587s"
Jul 30 03:45:04.460259 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.460191    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:04.460409 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:04.460318    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:04.467934 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.467898    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6887b" event={"ID":"79c63a29-b6fa-4412-9446-79a29e51deef","Type":"ContainerStarted","Data":"1ee50abe7c8cee4e4af82ca035daa4caf093a1f892594e57eb5a02c1ce640c30"}
Jul 30 03:45:04.469090 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.469056    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zdwwk" event={"ID":"db777952-f0cd-4473-8c10-5ad3c4d1af11","Type":"ContainerStarted","Data":"88b83d3be90fbbbec032d77b775eb7b565a75aeaaf8dfa2be35e56a6bd378d37"}
Jul 30 03:45:04.470196 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.470163    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fxtxs" event={"ID":"80d2e732-5a45-4f32-bec1-768138b5d8ae","Type":"ContainerStarted","Data":"16f2052973b81990e6a76c0880181290ec661c0c3fdedbbfb1139706a30b8f45"}
Jul 30 03:45:04.471408 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.471381    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbpx8" event={"ID":"621e8000-b5ef-4835-ad07-5fa5cd6e34e5","Type":"ContainerStarted","Data":"2a7a80e6008ccd417d2a427e14c51fce37f3fd40123858897945615aa3800bbe"}
Jul 30 03:45:04.472585 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.472543    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz" event={"ID":"3e669450-29a8-429b-abeb-31f57c441dda","Type":"ContainerStarted","Data":"826b67c7078a1bdf699680a523f6a7f17d894bf9519de7ba3572cab5368278b5"}
Jul 30 03:45:04.473722 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.473688    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5dzsm" event={"ID":"4b900747-00b2-49fe-8af7-ab67f091664c","Type":"ContainerStarted","Data":"e4a6ca140f60808210af41bf62034b14294c3b14a349b192646d67fa2057edc3"}
Jul 30 03:45:04.475001 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.474978    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sll89" event={"ID":"dd18769b-ddef-4460-980b-528c13aa0040","Type":"ContainerStarted","Data":"e5a452a1e72a2158afe7f6eb8452ce964d753fdb648a07b27a8b188fdf5ac9fc"}
Jul 30 03:45:04.476270 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.476242    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d6x52" event={"ID":"1b178858-acff-4796-87a6-9852811da3c6","Type":"ContainerStarted","Data":"9b3e76d6c9effd1f6dc09670049d025aefa59b657e681649104a727eaf4c4ca8"}
Jul 30 03:45:04.477554 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:04.477515    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z8gsc" event={"ID":"6583ab82-829c-4f0e-829f-0c7ccc835aa4","Type":"ContainerStarted","Data":"8dffe05aa67cb21d5d181d4454f13ddf025e3d53b4d43115db70dcb63a7ac917"}
Jul 30 03:45:05.081347 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:05.081305    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:05.081584 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:05.081375    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs\") pod \"network-metrics-daemon-nsf2h\" (UID: \"30c27db8-8cfe-492d-855d-4e085771ed3e\") " pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:05.081584 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:05.081533    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:45:05.081702 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:05.081616    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs podName:30c27db8-8cfe-492d-855d-4e085771ed3e nodeName:}" failed. No retries permitted until 2026-07-30 03:45:07.081593969 +0000 UTC m=+6.120304972 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs") pod "network-metrics-daemon-nsf2h" (UID: "30c27db8-8cfe-492d-855d-4e085771ed3e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:45:05.082067 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:05.082044    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 03:45:05.082128 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:05.082104    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret podName:a26002ce-a680-4af4-9513-044a00d2393d nodeName:}" failed. No retries permitted until 2026-07-30 03:45:07.082088739 +0000 UTC m=+6.120799742 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret") pod "global-pull-secret-syncer-vxddq" (UID: "a26002ce-a680-4af4-9513-044a00d2393d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 03:45:05.182445 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:05.182404    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cmgt\" (UniqueName: \"kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt\") pod \"network-check-target-bpwlj\" (UID: \"0295adc4-b5c7-42c2-ae4c-b653ae27ca5c\") " pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:05.182659 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:05.182634    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 03:45:05.182659 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:05.182656    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 03:45:05.182785 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:05.182669    2569 projected.go:196] Error preparing data for projected volume kube-api-access-5cmgt for pod openshift-network-diagnostics/network-check-target-bpwlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:45:05.182785 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:05.182728    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt podName:0295adc4-b5c7-42c2-ae4c-b653ae27ca5c nodeName:}" failed. No retries permitted until 2026-07-30 03:45:07.182710275 +0000 UTC m=+6.221421289 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5cmgt" (UniqueName: "kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt") pod "network-check-target-bpwlj" (UID: "0295adc4-b5c7-42c2-ae4c-b653ae27ca5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:45:05.403008 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:05.402857    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 03:40:04 +0000 UTC" deadline="2028-02-05 06:03:04.944770638 +0000 UTC"
Jul 30 03:45:05.403008 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:05.402957    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13322h17m59.541819184s"
Jul 30 03:45:05.463051 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:05.462305    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:05.463051 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:05.462446    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:05.463051 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:05.462888    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:05.463051 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:05.462990    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:06.460254 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:06.459992    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:06.460740 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:06.460370    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:06.515708 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:06.512653    2569 generic.go:356] "Generic (PLEG): container finished" podID="e735cf5928c25006e73eda857bc02787" containerID="884af01afc4c440ebd88357192925ee60b690701589a86b941ad33436401418f" exitCode=0
Jul 30 03:45:06.515708 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:06.512805    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" event={"ID":"e735cf5928c25006e73eda857bc02787","Type":"ContainerDied","Data":"884af01afc4c440ebd88357192925ee60b690701589a86b941ad33436401418f"}
Jul 30 03:45:06.534430 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:06.534398    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal" event={"ID":"9ae53f74fb865d246d654758e9704219","Type":"ContainerStarted","Data":"bae160264267e9bbd53f86b26feb272c051ffe3e6e80c7dcf6e122570f40410a"}
Jul 30 03:45:07.100268 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:07.100228    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:07.100469 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:07.100297    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs\") pod \"network-metrics-daemon-nsf2h\" (UID: \"30c27db8-8cfe-492d-855d-4e085771ed3e\") " pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:07.100542 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:07.100498    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:45:07.100598 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:07.100561    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs podName:30c27db8-8cfe-492d-855d-4e085771ed3e nodeName:}" failed. No retries permitted until 2026-07-30 03:45:11.100540777 +0000 UTC m=+10.139251782 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs") pod "network-metrics-daemon-nsf2h" (UID: "30c27db8-8cfe-492d-855d-4e085771ed3e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:45:07.100692 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:07.100645    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 03:45:07.100692 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:07.100683    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret podName:a26002ce-a680-4af4-9513-044a00d2393d nodeName:}" failed. No retries permitted until 2026-07-30 03:45:11.100669624 +0000 UTC m=+10.139380627 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret") pod "global-pull-secret-syncer-vxddq" (UID: "a26002ce-a680-4af4-9513-044a00d2393d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 03:45:07.201550 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:07.201511    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cmgt\" (UniqueName: \"kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt\") pod \"network-check-target-bpwlj\" (UID: \"0295adc4-b5c7-42c2-ae4c-b653ae27ca5c\") " pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:07.201773 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:07.201749    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 03:45:07.201841 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:07.201782    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 03:45:07.201841 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:07.201796    2569 projected.go:196] Error preparing data for projected volume kube-api-access-5cmgt for pod openshift-network-diagnostics/network-check-target-bpwlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:45:07.201963 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:07.201858    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt podName:0295adc4-b5c7-42c2-ae4c-b653ae27ca5c nodeName:}" failed. No retries permitted until 2026-07-30 03:45:11.201836743 +0000 UTC m=+10.240547744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5cmgt" (UniqueName: "kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt") pod "network-check-target-bpwlj" (UID: "0295adc4-b5c7-42c2-ae4c-b653ae27ca5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:45:07.460665 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:07.459687    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:07.460665 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:07.459842    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:07.460665 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:07.460542    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:07.460665 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:07.460659    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:07.574002 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:07.573963    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" event={"ID":"e735cf5928c25006e73eda857bc02787","Type":"ContainerStarted","Data":"3afc6ff22c6f8d026d606cb78324fa6d9933ba336dcd778bcd7f95a8f209b6b2"}
Jul 30 03:45:07.591879 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:07.591794    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal" podStartSLOduration=4.591773376 podStartE2EDuration="4.591773376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:45:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:06.548397575 +0000 UTC m=+5.587108599" watchObservedRunningTime="2026-07-30 03:45:07.591773376 +0000 UTC m=+6.630484400"
Jul 30 03:45:08.460092 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:08.460056    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:08.460278 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:08.460193    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:09.460801 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:09.460722    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:09.461260 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:09.460726    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:09.461260 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:09.460850    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:09.461260 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:09.460925    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:10.459896 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:10.459857    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:10.460074 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:10.460001    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:11.135986 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:11.135936    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:11.136407 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:11.136005    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs\") pod \"network-metrics-daemon-nsf2h\" (UID: \"30c27db8-8cfe-492d-855d-4e085771ed3e\") " pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:11.136407 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:11.136151    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:45:11.136407 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:11.136213    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs podName:30c27db8-8cfe-492d-855d-4e085771ed3e nodeName:}" failed. No retries permitted until 2026-07-30 03:45:19.136194622 +0000 UTC m=+18.174905625 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs") pod "network-metrics-daemon-nsf2h" (UID: "30c27db8-8cfe-492d-855d-4e085771ed3e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:45:11.136730 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:11.136636    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 03:45:11.136730 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:11.136696    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret podName:a26002ce-a680-4af4-9513-044a00d2393d nodeName:}" failed. No retries permitted until 2026-07-30 03:45:19.136678998 +0000 UTC m=+18.175390002 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret") pod "global-pull-secret-syncer-vxddq" (UID: "a26002ce-a680-4af4-9513-044a00d2393d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 03:45:11.237000 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:11.236959    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cmgt\" (UniqueName: \"kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt\") pod \"network-check-target-bpwlj\" (UID: \"0295adc4-b5c7-42c2-ae4c-b653ae27ca5c\") " pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:11.237231 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:11.237204    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 03:45:11.237334 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:11.237241    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 03:45:11.237334 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:11.237258    2569 projected.go:196] Error preparing data for projected volume kube-api-access-5cmgt for pod openshift-network-diagnostics/network-check-target-bpwlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:45:11.237334 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:11.237326    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt podName:0295adc4-b5c7-42c2-ae4c-b653ae27ca5c nodeName:}" failed. No retries permitted until 2026-07-30 03:45:19.237304107 +0000 UTC m=+18.276015123 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5cmgt" (UniqueName: "kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt") pod "network-check-target-bpwlj" (UID: "0295adc4-b5c7-42c2-ae4c-b653ae27ca5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:45:11.460219 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:11.460124    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:11.460357 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:11.460262    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:11.460710 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:11.460684    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:11.460850 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:11.460823    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:12.460117 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:12.460047    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:12.460590 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:12.460210    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:13.460298 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:13.459740    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:13.460298 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:13.459870    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:13.460298 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:13.459740    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:13.460298 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:13.460257    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:14.459950 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:14.459907    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:14.460182 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:14.460046    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:15.460057 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:15.460025    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:15.460525 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:15.460025    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:15.460525 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:15.460162    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:15.460525 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:15.460198    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:16.459890 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:16.459804    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:16.460031 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:16.459930    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:17.459967 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:17.459927    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:17.460395 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:17.460065    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:17.460395 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:17.460126    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:17.460395 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:17.460240    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:18.459807 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:18.459772    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:18.460050 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:18.459876    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:19.200844 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:19.200805    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:19.200844 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:19.200851    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs\") pod \"network-metrics-daemon-nsf2h\" (UID: \"30c27db8-8cfe-492d-855d-4e085771ed3e\") " pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:19.201046 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:19.200972    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:45:19.201046 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:19.200974    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 03:45:19.201046 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:19.201016    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs podName:30c27db8-8cfe-492d-855d-4e085771ed3e nodeName:}" failed. No retries permitted until 2026-07-30 03:45:35.201002822 +0000 UTC m=+34.239713818 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs") pod "network-metrics-daemon-nsf2h" (UID: "30c27db8-8cfe-492d-855d-4e085771ed3e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:45:19.201046 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:19.201031    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret podName:a26002ce-a680-4af4-9513-044a00d2393d nodeName:}" failed. No retries permitted until 2026-07-30 03:45:35.201023093 +0000 UTC m=+34.239734090 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret") pod "global-pull-secret-syncer-vxddq" (UID: "a26002ce-a680-4af4-9513-044a00d2393d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 03:45:19.301822 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:19.301790    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cmgt\" (UniqueName: \"kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt\") pod \"network-check-target-bpwlj\" (UID: \"0295adc4-b5c7-42c2-ae4c-b653ae27ca5c\") " pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:19.301950 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:19.301934    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 03:45:19.301950 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:19.301949    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 03:45:19.302034 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:19.301958    2569 projected.go:196] Error preparing data for projected volume kube-api-access-5cmgt for pod openshift-network-diagnostics/network-check-target-bpwlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:45:19.302034 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:19.302012    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt podName:0295adc4-b5c7-42c2-ae4c-b653ae27ca5c nodeName:}" failed. No retries permitted until 2026-07-30 03:45:35.301994603 +0000 UTC m=+34.340705602 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5cmgt" (UniqueName: "kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt") pod "network-check-target-bpwlj" (UID: "0295adc4-b5c7-42c2-ae4c-b653ae27ca5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:45:19.459460 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:19.459374    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:19.459619 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:19.459507    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:19.459619 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:19.459564    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:19.459725 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:19.459641    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:20.459491 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:20.459460    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:20.459930 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:20.459611    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:21.460068 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:21.460028    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:21.460461 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:21.460098    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:21.460461 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:21.460184    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:21.460461 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:21.460338    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:22.459594 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:22.459564    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:22.459755 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:22.459668    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:23.459465 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:23.459438    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:23.459945 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:23.459591    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:23.459945 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:23.459650    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:23.459945 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:23.459751    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:24.459366 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.459258    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:24.459502 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:24.459477    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:24.611389 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.611156    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sll89" event={"ID":"dd18769b-ddef-4460-980b-528c13aa0040","Type":"ContainerStarted","Data":"5a61c20da90b80794219c97ea7ee6b211bf4c6c618e8b4ba25697afc06b2f22a"}
Jul 30 03:45:24.612582 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.612557    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d6x52" event={"ID":"1b178858-acff-4796-87a6-9852811da3c6","Type":"ContainerStarted","Data":"2dfeea77ddd4142032b80db7fae3632d64f3d0b57b7ea27792b8e6622e1df3c6"}
Jul 30 03:45:24.614674 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.614654    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6887b" event={"ID":"79c63a29-b6fa-4412-9446-79a29e51deef","Type":"ContainerStarted","Data":"c9ff89eb0ad83a03064d656831e4ddcf920be55ffe227145276dd70261bd095a"}
Jul 30 03:45:24.614740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.614720    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6887b" event={"ID":"79c63a29-b6fa-4412-9446-79a29e51deef","Type":"ContainerStarted","Data":"a36f2df520a8784e54733b637c54c9d0f04816e7ae944e66835a229861b588d7"}
Jul 30 03:45:24.614781 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.614763    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6887b" event={"ID":"79c63a29-b6fa-4412-9446-79a29e51deef","Type":"ContainerStarted","Data":"68e3ff45e78cbf3aeb876a7a2c1de3510fae10db3f0ed5eac484a26cefa76ccf"}
Jul 30 03:45:24.616026 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.615994    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zdwwk" event={"ID":"db777952-f0cd-4473-8c10-5ad3c4d1af11","Type":"ContainerStarted","Data":"7e616e0c377945fff7e86f30065fccf317b863873fbe4c63996091a7a33e3166"}
Jul 30 03:45:24.617470 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.617442    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fxtxs" event={"ID":"80d2e732-5a45-4f32-bec1-768138b5d8ae","Type":"ContainerStarted","Data":"6f125b63f93209236b00736da38e85c212181bbe11589fb7a8530c09762cc130"}
Jul 30 03:45:24.618950 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.618925    2569 generic.go:356] "Generic (PLEG): container finished" podID="621e8000-b5ef-4835-ad07-5fa5cd6e34e5" containerID="40e78658df2b66224123758e85057b4c5663fa3124cc4b36ac4a8d58d270389c" exitCode=0
Jul 30 03:45:24.619033 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.618997    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbpx8" event={"ID":"621e8000-b5ef-4835-ad07-5fa5cd6e34e5","Type":"ContainerDied","Data":"40e78658df2b66224123758e85057b4c5663fa3124cc4b36ac4a8d58d270389c"}
Jul 30 03:45:24.620295 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.620276    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz" event={"ID":"3e669450-29a8-429b-abeb-31f57c441dda","Type":"ContainerStarted","Data":"60f751f71f20d1c7afbc0498de2db773f048e622104e280922d587da569a70c4"}
Jul 30 03:45:24.621636 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.621616    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5dzsm" event={"ID":"4b900747-00b2-49fe-8af7-ab67f091664c","Type":"ContainerStarted","Data":"9b2463c577efa7f1b8a2593dba339562b383241149a53c60fe5cbc71e10b5010"}
Jul 30 03:45:24.625792 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.625756    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" podStartSLOduration=21.6257458 podStartE2EDuration="21.6257458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:45:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:07.592164583 +0000 UTC m=+6.630875607" watchObservedRunningTime="2026-07-30 03:45:24.6257458 +0000 UTC m=+23.664456822"
Jul 30 03:45:24.626004 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.625986    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sll89" podStartSLOduration=2.147412257 podStartE2EDuration="22.625980184s" totalImagesPullingTime="20.478567927s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:24.625463422 +0000 UTC m=+23.664174440" watchObservedRunningTime="2026-07-30 03:45:24.625980184 +0000 UTC m=+23.664691252"
Jul 30 03:45:24.637848 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.637816    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5dzsm" podStartSLOduration=2.169288729 podStartE2EDuration="22.637805143s" totalImagesPullingTime="20.468516414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:24.637327719 +0000 UTC m=+23.676038738" watchObservedRunningTime="2026-07-30 03:45:24.637805143 +0000 UTC m=+23.676516161"
Jul 30 03:45:24.680577 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.680521    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zdwwk" podStartSLOduration=2.190719586 podStartE2EDuration="22.680509251s" totalImagesPullingTime="20.489789665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:24.680176186 +0000 UTC m=+23.718887204" watchObservedRunningTime="2026-07-30 03:45:24.680509251 +0000 UTC m=+23.719220270"
Jul 30 03:45:24.696431 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.696397    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fxtxs" podStartSLOduration=2.220700569 podStartE2EDuration="22.696385515s" totalImagesPullingTime="20.475684946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:24.695921583 +0000 UTC m=+23.734632601" watchObservedRunningTime="2026-07-30 03:45:24.696385515 +0000 UTC m=+23.735096533"
Jul 30 03:45:24.710757 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:24.710726    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d6x52" podStartSLOduration=2.141935049 podStartE2EDuration="22.71071314s" totalImagesPullingTime="20.568778091s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:24.710659287 +0000 UTC m=+23.749370306" watchObservedRunningTime="2026-07-30 03:45:24.71071314 +0000 UTC m=+23.749424162"
Jul 30 03:45:25.459488 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:25.459458    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:25.459670 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:25.459489    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:25.459670 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:25.459626    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:25.459989 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:25.459732    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:25.627016 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:25.626977    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6887b" event={"ID":"79c63a29-b6fa-4412-9446-79a29e51deef","Type":"ContainerStarted","Data":"98f694dc2e0a6fa458a1fb5524f225fec333435e6d3187770507148452bec60d"}
Jul 30 03:45:25.627200 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:25.627081    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6887b" event={"ID":"79c63a29-b6fa-4412-9446-79a29e51deef","Type":"ContainerStarted","Data":"d8f2fd82f66e81c0f7ef32cc4b7dec86f4e9c32484ceec24c2693d24aee26e5d"}
Jul 30 03:45:25.627200 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:25.627139    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6887b" event={"ID":"79c63a29-b6fa-4412-9446-79a29e51deef","Type":"ContainerStarted","Data":"ba56fd64186944fa0cfe87c71ffe3a594c63cf295165b6c442cb9f34111c5ced"}
Jul 30 03:45:25.826937 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:25.826895    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 03:45:26.430203 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:26.430109    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T03:45:25.826914555Z","UUID":"bea8a4b4-08ac-4513-b4da-76a363a077a5","Handler":null,"Name":"","Endpoint":""}
Jul 30 03:45:26.431810 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:26.431784    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 03:45:26.431940 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:26.431830    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 03:45:26.459707 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:26.459681    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:26.460161 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:26.459800    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:26.631202 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:26.631162    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z8gsc" event={"ID":"6583ab82-829c-4f0e-829f-0c7ccc835aa4","Type":"ContainerStarted","Data":"0409e62ad96daeef02f501be6ce3b53e1651ebd88a6c20e5fe78de5661c04e2f"}
Jul 30 03:45:26.634060 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:26.634029    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz" event={"ID":"3e669450-29a8-429b-abeb-31f57c441dda","Type":"ContainerStarted","Data":"86d10062a812a5689e9b3c9a263e394a9b1f44cf9f353307da7c51ec97442f53"}
Jul 30 03:45:26.645589 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:26.645522    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z8gsc" podStartSLOduration=4.147466261 podStartE2EDuration="24.645505706s" totalImagesPullingTime="20.498039445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:26.644990313 +0000 UTC m=+25.683701333" watchObservedRunningTime="2026-07-30 03:45:26.645505706 +0000 UTC m=+25.684216727"
Jul 30 03:45:26.882267 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:26.882230    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fxtxs"
Jul 30 03:45:26.883057 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:26.883037    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fxtxs"
Jul 30 03:45:27.459798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:27.459725    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:27.459798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:27.459752    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:27.460169 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:27.459861    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:27.460169 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:27.459955    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:27.640252 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:27.640204    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6887b" event={"ID":"79c63a29-b6fa-4412-9446-79a29e51deef","Type":"ContainerStarted","Data":"9c87207942e6e1d0d011ee35c4a1fc41d68829007d1085c44eca32e99cbfea28"}
Jul 30 03:45:27.640506 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:27.640481    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fxtxs"
Jul 30 03:45:27.641135 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:27.641113    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fxtxs"
Jul 30 03:45:28.460075 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:28.460047    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:28.460458 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:28.460145    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:29.459501 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:29.459318    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:29.459635 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:29.459391    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:29.459683 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:29.459630    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:29.459772 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:29.459753    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:29.648281 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:29.648205    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6887b" event={"ID":"79c63a29-b6fa-4412-9446-79a29e51deef","Type":"ContainerStarted","Data":"89c1cbad59a6531ad885e615bb7b8454ca7f3f15426daf764953cb6c775a29ad"}
Jul 30 03:45:29.649111 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:29.648455    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:29.650167 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:29.650142    2569 generic.go:356] "Generic (PLEG): container finished" podID="621e8000-b5ef-4835-ad07-5fa5cd6e34e5" containerID="a1632e233d783a01dc733c89cf8a073c29a3de2cbf68dfc8f5aba86bce7e2e06" exitCode=0
Jul 30 03:45:29.650271 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:29.650225    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbpx8" event={"ID":"621e8000-b5ef-4835-ad07-5fa5cd6e34e5","Type":"ContainerDied","Data":"a1632e233d783a01dc733c89cf8a073c29a3de2cbf68dfc8f5aba86bce7e2e06"}
Jul 30 03:45:29.652380 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:29.652344    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz" event={"ID":"3e669450-29a8-429b-abeb-31f57c441dda","Type":"ContainerStarted","Data":"90ae5c53e6a4372f2673dc9861ab73035798f628aaae9540659795342726d432"}
Jul 30 03:45:29.665207 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:29.665179    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:29.673023 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:29.672988    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6887b" podStartSLOduration=7.185432974 podStartE2EDuration="27.672977278s" totalImagesPullingTime="20.487544304s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:29.672782623 +0000 UTC m=+28.711493644" watchObservedRunningTime="2026-07-30 03:45:29.672977278 +0000 UTC m=+28.711688296"
Jul 30 03:45:29.688415 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:29.688380    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gftvz" podStartSLOduration=2.669223013 podStartE2EDuration="27.688369163s" totalImagesPullingTime="25.01914615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:45:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:29.688164312 +0000 UTC m=+28.726875332" watchObservedRunningTime="2026-07-30 03:45:29.688369163 +0000 UTC m=+28.727080182"
Jul 30 03:45:30.459990 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:30.459779    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:30.460116 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:30.460026    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:30.655198 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:30.655173    2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 30 03:45:30.655755 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:30.655732    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:30.672864 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:30.672825    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:30.674064 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:30.674042    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:45:31.460688 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:31.460655    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:31.460907 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:31.460736    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:31.460907 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:31.460787    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:31.460907 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:31.460876    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:31.597340 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:31.597305    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bpwlj"]
Jul 30 03:45:31.600412 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:31.600384    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vxddq"]
Jul 30 03:45:31.600593 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:31.600474    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:31.600659 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:31.600587    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:31.601154 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:31.601118    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nsf2h"]
Jul 30 03:45:31.658905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:31.658875    2569 generic.go:356] "Generic (PLEG): container finished" podID="621e8000-b5ef-4835-ad07-5fa5cd6e34e5" containerID="0bac6eadb8ed0e90b155431fe7f281dc4b134b137a1c34cf0dc1390d1941e849" exitCode=0
Jul 30 03:45:31.659248 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:31.658941    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbpx8" event={"ID":"621e8000-b5ef-4835-ad07-5fa5cd6e34e5","Type":"ContainerDied","Data":"0bac6eadb8ed0e90b155431fe7f281dc4b134b137a1c34cf0dc1390d1941e849"}
Jul 30 03:45:31.659248 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:31.658963    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:31.659248 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:31.659047    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:31.659248 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:31.659046    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:31.659460 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:31.659436    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:33.459522 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:33.459494    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:33.459931 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:33.459505    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:33.459931 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:33.459658    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:33.459931 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:33.459705    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:33.459931 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:33.459801    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:33.459931 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:33.459890    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:33.667996 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:33.667968    2569 generic.go:356] "Generic (PLEG): container finished" podID="621e8000-b5ef-4835-ad07-5fa5cd6e34e5" containerID="3a32540711b50717bca4a053d87c97f9265834f206df8bfcd02ee59145d3ea6b" exitCode=0
Jul 30 03:45:33.668126 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:33.668011    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbpx8" event={"ID":"621e8000-b5ef-4835-ad07-5fa5cd6e34e5","Type":"ContainerDied","Data":"3a32540711b50717bca4a053d87c97f9265834f206df8bfcd02ee59145d3ea6b"}
Jul 30 03:45:35.227309 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:35.227273    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs\") pod \"network-metrics-daemon-nsf2h\" (UID: \"30c27db8-8cfe-492d-855d-4e085771ed3e\") " pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:35.227751 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:35.227386    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:35.227751 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:35.227423    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:45:35.227751 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:35.227493    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs podName:30c27db8-8cfe-492d-855d-4e085771ed3e nodeName:}" failed. No retries permitted until 2026-07-30 03:46:07.227477426 +0000 UTC m=+66.266188424 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs") pod "network-metrics-daemon-nsf2h" (UID: "30c27db8-8cfe-492d-855d-4e085771ed3e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:45:35.227751 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:35.227511    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 03:45:35.227751 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:35.227583    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret podName:a26002ce-a680-4af4-9513-044a00d2393d nodeName:}" failed. No retries permitted until 2026-07-30 03:46:07.227567086 +0000 UTC m=+66.266278085 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret") pod "global-pull-secret-syncer-vxddq" (UID: "a26002ce-a680-4af4-9513-044a00d2393d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 03:45:35.328350 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:35.328268    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cmgt\" (UniqueName: \"kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt\") pod \"network-check-target-bpwlj\" (UID: \"0295adc4-b5c7-42c2-ae4c-b653ae27ca5c\") " pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:35.328500 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:35.328405    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 03:45:35.328500 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:35.328421    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 03:45:35.328500 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:35.328444    2569 projected.go:196] Error preparing data for projected volume kube-api-access-5cmgt for pod openshift-network-diagnostics/network-check-target-bpwlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:45:35.328632 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:35.328503    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt podName:0295adc4-b5c7-42c2-ae4c-b653ae27ca5c nodeName:}" failed. No retries permitted until 2026-07-30 03:46:07.328485386 +0000 UTC m=+66.367196383 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5cmgt" (UniqueName: "kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt") pod "network-check-target-bpwlj" (UID: "0295adc4-b5c7-42c2-ae4c-b653ae27ca5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:45:35.459778 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:35.459743    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:35.459953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:35.459860    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:35.459953 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:35.459874    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxddq" podUID="a26002ce-a680-4af4-9513-044a00d2393d"
Jul 30 03:45:35.459953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:35.459750    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:35.459953 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:35.459941    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bpwlj" podUID="0295adc4-b5c7-42c2-ae4c-b653ae27ca5c"
Jul 30 03:45:35.460163 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:35.460016    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nsf2h" podUID="30c27db8-8cfe-492d-855d-4e085771ed3e"
Jul 30 03:45:36.760232 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.759995    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeReady"
Jul 30 03:45:36.760607 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.760379    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 03:45:36.794943 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.794414    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"]
Jul 30 03:45:36.798811 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.798790    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:36.800889 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.800869    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 30 03:45:36.801183 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.801167    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tfs77\""
Jul 30 03:45:36.801648 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.801628    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 30 03:45:36.801855 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.801840    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 30 03:45:36.806019 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.805997    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 30 03:45:36.807436 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.807410    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lzf7c"]
Jul 30 03:45:36.810691 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.810614    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nth9r"]
Jul 30 03:45:36.810784 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.810720    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lzf7c"
Jul 30 03:45:36.812892 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.812871    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vxgj5\""
Jul 30 03:45:36.812892 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.812886    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 03:45:36.812995 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.812917    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 03:45:36.813042 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.812872    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 03:45:36.814314 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.813804    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"]
Jul 30 03:45:36.814314 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.813857    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:36.815784 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.815720    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 03:45:36.815784 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.815755    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5s29r\""
Jul 30 03:45:36.815925 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.815855    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 03:45:36.820766 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.820748    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nth9r"]
Jul 30 03:45:36.820846 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.820787    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lzf7c"]
Jul 30 03:45:36.935999 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.935968    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:36.939820 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.939800    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-bound-sa-token\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:36.939944 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.939832    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqh82\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-kube-api-access-fqh82\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:36.939944 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.939858    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad3a8ea4-d784-43ae-832c-16a6ebf99c1f-cert\") pod \"ingress-canary-lzf7c\" (UID: \"ad3a8ea4-d784-43ae-832c-16a6ebf99c1f\") " pod="openshift-ingress-canary/ingress-canary-lzf7c"
Jul 30 03:45:36.939944 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.939886    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwhrr\" (UniqueName: \"kubernetes.io/projected/9d01562f-999a-4b59-b5fa-7d6738df7633-kube-api-access-zwhrr\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:36.939944 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.939905    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-ca-trust-extracted\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:36.939944 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.939919    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-installation-pull-secrets\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:36.940171 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.939944    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-trusted-ca\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:36.940171 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.940048    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-certificates\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:36.940171 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.940071    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq2t6\" (UniqueName: \"kubernetes.io/projected/ad3a8ea4-d784-43ae-832c-16a6ebf99c1f-kube-api-access-zq2t6\") pod \"ingress-canary-lzf7c\" (UID: \"ad3a8ea4-d784-43ae-832c-16a6ebf99c1f\") " pod="openshift-ingress-canary/ingress-canary-lzf7c"
Jul 30 03:45:36.940171 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.940111    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9d01562f-999a-4b59-b5fa-7d6738df7633-config-volume\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:36.940171 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.940134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:36.940340 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.940182    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-image-registry-private-configuration\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:36.940340 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.940203    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d01562f-999a-4b59-b5fa-7d6738df7633-tmp-dir\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:36.940340 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:36.940225    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.013869 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.013774    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:37.040754 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.040723    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9d01562f-999a-4b59-b5fa-7d6738df7633-config-volume\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:37.040907 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.040764    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:37.040968 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:37.040908    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:45:37.040968 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.040929    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-image-registry-private-configuration\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.040968 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:37.040964    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls podName:9d01562f-999a-4b59-b5fa-7d6738df7633 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:37.540945785 +0000 UTC m=+36.579656782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls") pod "dns-default-nth9r" (UID: "9d01562f-999a-4b59-b5fa-7d6738df7633") : secret "dns-default-metrics-tls" not found
Jul 30 03:45:37.041138 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.040998    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d01562f-999a-4b59-b5fa-7d6738df7633-tmp-dir\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:37.041138 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.041028    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.041138 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.041045    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-bound-sa-token\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.041138 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.041073    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqh82\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-kube-api-access-fqh82\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.041138 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.041093    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad3a8ea4-d784-43ae-832c-16a6ebf99c1f-cert\") pod \"ingress-canary-lzf7c\" (UID: \"ad3a8ea4-d784-43ae-832c-16a6ebf99c1f\") " pod="openshift-ingress-canary/ingress-canary-lzf7c"
Jul 30 03:45:37.041138 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.041110    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwhrr\" (UniqueName: \"kubernetes.io/projected/9d01562f-999a-4b59-b5fa-7d6738df7633-kube-api-access-zwhrr\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:37.041138 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.041131    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-ca-trust-extracted\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.041438 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.041156    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-installation-pull-secrets\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.041438 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.041180    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-trusted-ca\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.041438 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.041221    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-certificates\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.041438 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.041249    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq2t6\" (UniqueName: \"kubernetes.io/projected/ad3a8ea4-d784-43ae-832c-16a6ebf99c1f-kube-api-access-zq2t6\") pod \"ingress-canary-lzf7c\" (UID: \"ad3a8ea4-d784-43ae-832c-16a6ebf99c1f\") " pod="openshift-ingress-canary/ingress-canary-lzf7c"
Jul 30 03:45:37.041438 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.041368    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9d01562f-999a-4b59-b5fa-7d6738df7633-config-volume\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:37.047560 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:37.043635    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 03:45:37.047560 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:37.043658    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz: secret "image-registry-tls" not found
Jul 30 03:45:37.047560 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:37.043712    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls podName:9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca nodeName:}" failed. No retries permitted until 2026-07-30 03:45:37.543698373 +0000 UTC m=+36.582409387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls") pod "image-registry-6f9c8fcc6c-hpvwz" (UID: "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca") : secret "image-registry-tls" not found
Jul 30 03:45:37.047560 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.044206    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-certificates\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.047560 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.044312    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-ca-trust-extracted\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.047560 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.045612    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d01562f-999a-4b59-b5fa-7d6738df7633-tmp-dir\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:37.047560 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.046264    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-image-registry-private-configuration\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.047560 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.046281    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad3a8ea4-d784-43ae-832c-16a6ebf99c1f-cert\") pod \"ingress-canary-lzf7c\" (UID: \"ad3a8ea4-d784-43ae-832c-16a6ebf99c1f\") " pod="openshift-ingress-canary/ingress-canary-lzf7c"
Jul 30 03:45:37.047560 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.046435    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-trusted-ca\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.048035 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.047829    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-installation-pull-secrets\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.050409 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.050388    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwhrr\" (UniqueName: \"kubernetes.io/projected/9d01562f-999a-4b59-b5fa-7d6738df7633-kube-api-access-zwhrr\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:37.050554 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.050472    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq2t6\" (UniqueName: \"kubernetes.io/projected/ad3a8ea4-d784-43ae-832c-16a6ebf99c1f-kube-api-access-zq2t6\") pod \"ingress-canary-lzf7c\" (UID: \"ad3a8ea4-d784-43ae-832c-16a6ebf99c1f\") " pod="openshift-ingress-canary/ingress-canary-lzf7c"
Jul 30 03:45:37.053354 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.053333    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-bound-sa-token\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.053629 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.053594    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqh82\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-kube-api-access-fqh82\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.122274 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.122239    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lzf7c"
Jul 30 03:45:37.459564 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.459508    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:45:37.459841 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.459526    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:45:37.460284 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.459593    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:45:37.462390 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.462372    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 03:45:37.463066 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.463046    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 03:45:37.463066 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.463053    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 03:45:37.463213 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.463104    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 03:45:37.463213 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.463048    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kzw94\""
Jul 30 03:45:37.463213 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.463182    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hmgxg\""
Jul 30 03:45:37.544997 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.544958    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:37.545146 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.545029    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:37.545146 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:37.545116    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:45:37.545255 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:37.545187    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls podName:9d01562f-999a-4b59-b5fa-7d6738df7633 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:38.545170692 +0000 UTC m=+37.583881710 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls") pod "dns-default-nth9r" (UID: "9d01562f-999a-4b59-b5fa-7d6738df7633") : secret "dns-default-metrics-tls" not found
Jul 30 03:45:37.545255 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:37.545193    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 03:45:37.545255 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:37.545211    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz: secret "image-registry-tls" not found
Jul 30 03:45:37.545371 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:37.545258    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls podName:9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca nodeName:}" failed. No retries permitted until 2026-07-30 03:45:38.545241282 +0000 UTC m=+37.583952281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls") pod "image-registry-6f9c8fcc6c-hpvwz" (UID: "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca") : secret "image-registry-tls" not found
Jul 30 03:45:37.931585 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:37.931562    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lzf7c"]
Jul 30 03:45:37.935035 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:37.935008    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad3a8ea4_d784_43ae_832c_16a6ebf99c1f.slice/crio-b889bb9b7160ba1c6697d4a228920e4bf6ece1a69c042a5ca4c81f0aaf51b6b8 WatchSource:0}: Error finding container b889bb9b7160ba1c6697d4a228920e4bf6ece1a69c042a5ca4c81f0aaf51b6b8: Status 404 returned error can't find the container with id b889bb9b7160ba1c6697d4a228920e4bf6ece1a69c042a5ca4c81f0aaf51b6b8
Jul 30 03:45:38.553391 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:38.553197    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:38.553556 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:38.553417    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:38.553556 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:38.553350    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:45:38.553556 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:38.553514    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls podName:9d01562f-999a-4b59-b5fa-7d6738df7633 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:40.55349959 +0000 UTC m=+39.592210587 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls") pod "dns-default-nth9r" (UID: "9d01562f-999a-4b59-b5fa-7d6738df7633") : secret "dns-default-metrics-tls" not found
Jul 30 03:45:38.553556 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:38.553519    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 03:45:38.553556 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:38.553548    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz: secret "image-registry-tls" not found
Jul 30 03:45:38.553725 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:38.553590    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls podName:9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca nodeName:}" failed. No retries permitted until 2026-07-30 03:45:40.553579057 +0000 UTC m=+39.592290054 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls") pod "image-registry-6f9c8fcc6c-hpvwz" (UID: "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca") : secret "image-registry-tls" not found
Jul 30 03:45:38.683793 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:38.683764    2569 generic.go:356] "Generic (PLEG): container finished" podID="621e8000-b5ef-4835-ad07-5fa5cd6e34e5" containerID="023a08fc76b7aa90f666edabeb565b6c3175d0095c63d6d2cf8e550ce7c87976" exitCode=0
Jul 30 03:45:38.683932 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:38.683846    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbpx8" event={"ID":"621e8000-b5ef-4835-ad07-5fa5cd6e34e5","Type":"ContainerDied","Data":"023a08fc76b7aa90f666edabeb565b6c3175d0095c63d6d2cf8e550ce7c87976"}
Jul 30 03:45:38.685188 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:38.685160    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lzf7c" event={"ID":"ad3a8ea4-d784-43ae-832c-16a6ebf99c1f","Type":"ContainerStarted","Data":"b889bb9b7160ba1c6697d4a228920e4bf6ece1a69c042a5ca4c81f0aaf51b6b8"}
Jul 30 03:45:39.689696 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:39.689665    2569 generic.go:356] "Generic (PLEG): container finished" podID="621e8000-b5ef-4835-ad07-5fa5cd6e34e5" containerID="33f84f86c713b6022b2ecc8345213995857f2dc08c505c57d5e162838234b6e5" exitCode=0
Jul 30 03:45:39.690046 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:39.689702    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbpx8" event={"ID":"621e8000-b5ef-4835-ad07-5fa5cd6e34e5","Type":"ContainerDied","Data":"33f84f86c713b6022b2ecc8345213995857f2dc08c505c57d5e162838234b6e5"}
Jul 30 03:45:40.221606 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:40.221562    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:40.271426 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:40.271400    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:40.568346 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:40.568309    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:40.568558 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:40.568382    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:40.568558 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:40.568463    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:45:40.568558 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:40.568516    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 03:45:40.568558 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:40.568559    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz: secret "image-registry-tls" not found
Jul 30 03:45:40.568740 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:40.568528    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls podName:9d01562f-999a-4b59-b5fa-7d6738df7633 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:44.568513834 +0000 UTC m=+43.607224831 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls") pod "dns-default-nth9r" (UID: "9d01562f-999a-4b59-b5fa-7d6738df7633") : secret "dns-default-metrics-tls" not found
Jul 30 03:45:40.568740 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:40.568641    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls podName:9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca nodeName:}" failed. No retries permitted until 2026-07-30 03:45:44.568622976 +0000 UTC m=+43.607333985 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls") pod "image-registry-6f9c8fcc6c-hpvwz" (UID: "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca") : secret "image-registry-tls" not found
Jul 30 03:45:40.696753 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:40.696713    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbpx8" event={"ID":"621e8000-b5ef-4835-ad07-5fa5cd6e34e5","Type":"ContainerStarted","Data":"990b1e1f68a974e4a72b9f55469c3103c6e9f743f93f3322058b4934782d50e7"}
Jul 30 03:45:40.718029 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:40.717980    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mbpx8" podStartSLOduration=7.050422016 podStartE2EDuration="38.717964503s" totalImagesPullingTime="31.667542487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:45:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:40.716341942 +0000 UTC m=+39.755052973" watchObservedRunningTime="2026-07-30 03:45:40.717964503 +0000 UTC m=+39.756675522"
Jul 30 03:45:40.837768 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:40.837673    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:40.938326 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:40.938287    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:41.372092 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.372067    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:41.472699 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.472666    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:41.632015 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.631914    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-tth9v"]
Jul 30 03:45:41.673072 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.673044    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-v7mn4"]
Jul 30 03:45:41.673214 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.673169    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:41.675145 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.675109    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 30 03:45:41.675145 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.675119    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 30 03:45:41.675325 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.675146    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-w8k5f\""
Jul 30 03:45:41.675325 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.675276    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 30 03:45:41.675943 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.675928    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 30 03:45:41.679801 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.679784    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 30 03:45:41.697041 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.697020    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-tth9v"]
Jul 30 03:45:41.697356 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.697048    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-v7mn4"]
Jul 30 03:45:41.697356 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.697111    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.699293 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.699269    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 30 03:45:41.699375 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.699303    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-bbddc\""
Jul 30 03:45:41.699564 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.699523    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 03:45:41.700855 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.700790    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 30 03:45:41.701958 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.701758    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 03:45:41.703439 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.703417    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lzf7c" event={"ID":"ad3a8ea4-d784-43ae-832c-16a6ebf99c1f","Type":"ContainerStarted","Data":"56a57bb0afc2a8c4965d971b73cc5ead376cb700465551d02d4223cb3acda4a4"}
Jul 30 03:45:41.706143 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.706122    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 30 03:45:41.726228 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.726098    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lzf7c" podStartSLOduration=2.371270114 podStartE2EDuration="5.726084631s" totalImagesPullingTime="3.354814517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:41.725501758 +0000 UTC m=+40.764212770" watchObservedRunningTime="2026-07-30 03:45:41.726084631 +0000 UTC m=+40.764795652"
Jul 30 03:45:41.730486 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.730144    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"]
Jul 30 03:45:41.750914 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.750891    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"]
Jul 30 03:45:41.751025 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.751009    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:45:41.755162 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.755145    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 30 03:45:41.755162 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.755160    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-c8qpq\""
Jul 30 03:45:41.755413 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.755402    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 03:45:41.756663 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.756648    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 03:45:41.759668 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.759650    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 30 03:45:41.763063 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.763048    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"]
Jul 30 03:45:41.763122 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.763074    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"]
Jul 30 03:45:41.763166 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.763134    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"
Jul 30 03:45:41.766918 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.766899    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tqpmr\""
Jul 30 03:45:41.767993 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.767978    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 30 03:45:41.768366 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.768291    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 30 03:45:41.778411 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.778390    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-tmp\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.778489 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.778423    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pjjq\" (UniqueName: \"kubernetes.io/projected/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-kube-api-access-5pjjq\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.778489 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.778445    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0-serving-cert\") pod \"console-operator-584dcb6b54-tth9v\" (UID: \"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0\") " pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:41.778489 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.778468    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8svg\" (UniqueName: \"kubernetes.io/projected/30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0-kube-api-access-h8svg\") pod \"console-operator-584dcb6b54-tth9v\" (UID: \"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0\") " pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:41.778665 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.778491    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-service-ca-bundle\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.778665 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.778598    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-serving-cert\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.778665 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.778621    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0-trusted-ca\") pod \"console-operator-584dcb6b54-tth9v\" (UID: \"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0\") " pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:41.778665 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.778640    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-snapshots\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.778665 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.778655    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.778818 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.778727    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0-config\") pod \"console-operator-584dcb6b54-tth9v\" (UID: \"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0\") " pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:41.879958 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.879932    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:45:41.880064 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.879996    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0-serving-cert\") pod \"console-operator-584dcb6b54-tth9v\" (UID: \"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0\") " pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:41.880064 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.880028    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-serving-cert\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.880064 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.880048    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:45:41.880217 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.880067    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-snapshots\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.880217 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.880089    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.880217 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.880120    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7lnp\" (UniqueName: \"kubernetes.io/projected/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-kube-api-access-l7lnp\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:45:41.880344 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.880237    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-tmp\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.880344 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.880269    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pjjq\" (UniqueName: \"kubernetes.io/projected/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-kube-api-access-5pjjq\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.880344 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.880310    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0-config\") pod \"console-operator-584dcb6b54-tth9v\" (UID: \"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0\") " pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:41.880344 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.880332    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-service-ca-bundle\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.880561 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.880380    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tcpmc\" (UID: \"56a7549c-0cff-4485-8518-30d431954079\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"
Jul 30 03:45:41.880561 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.880407    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8svg\" (UniqueName: \"kubernetes.io/projected/30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0-kube-api-access-h8svg\") pod \"console-operator-584dcb6b54-tth9v\" (UID: \"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0\") " pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:41.880561 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.880435    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/56a7549c-0cff-4485-8518-30d431954079-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-tcpmc\" (UID: \"56a7549c-0cff-4485-8518-30d431954079\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"
Jul 30 03:45:41.880561 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.880462    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0-trusted-ca\") pod \"console-operator-584dcb6b54-tth9v\" (UID: \"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0\") " pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:41.880734 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.880662    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-snapshots\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.881068 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.881043    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-service-ca-bundle\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.881197 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.881086    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.881197 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.881133    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-tmp\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.881455 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.881437    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0-trusted-ca\") pod \"console-operator-584dcb6b54-tth9v\" (UID: \"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0\") " pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:41.881499 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.881443    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0-config\") pod \"console-operator-584dcb6b54-tth9v\" (UID: \"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0\") " pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:41.884341 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.884300    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-serving-cert\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.884341 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.884330    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0-serving-cert\") pod \"console-operator-584dcb6b54-tth9v\" (UID: \"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0\") " pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:41.889167 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.889146    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pjjq\" (UniqueName: \"kubernetes.io/projected/e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2-kube-api-access-5pjjq\") pod \"insights-operator-5885fdb759-v7mn4\" (UID: \"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2\") " pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:41.889332 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.889313    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8svg\" (UniqueName: \"kubernetes.io/projected/30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0-kube-api-access-h8svg\") pod \"console-operator-584dcb6b54-tth9v\" (UID: \"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0\") " pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:41.891478 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.891456    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:41.953258 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.953232    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:41.981351 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.981331    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:41.981493 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.981475    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:45:41.981559 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.981505    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7lnp\" (UniqueName: \"kubernetes.io/projected/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-kube-api-access-l7lnp\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:45:41.981601 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.981559    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tcpmc\" (UID: \"56a7549c-0cff-4485-8518-30d431954079\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"
Jul 30 03:45:41.981601 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.981578    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/56a7549c-0cff-4485-8518-30d431954079-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-tcpmc\" (UID: \"56a7549c-0cff-4485-8518-30d431954079\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"
Jul 30 03:45:41.981662 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:41.981598    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 03:45:41.981662 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.981612    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:45:41.981662 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:41.981650    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls podName:e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:42.481632868 +0000 UTC m=+41.520343868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ltv96" (UID: "e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7") : secret "cluster-monitoring-operator-tls" not found
Jul 30 03:45:41.981772 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:41.981681    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 03:45:41.981772 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:41.981751    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert podName:56a7549c-0cff-4485-8518-30d431954079 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:42.481733985 +0000 UTC m=+41.520444982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-tcpmc" (UID: "56a7549c-0cff-4485-8518-30d431954079") : secret "networking-console-plugin-cert" not found
Jul 30 03:45:41.982118 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.982100    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/56a7549c-0cff-4485-8518-30d431954079-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-tcpmc\" (UID: \"56a7549c-0cff-4485-8518-30d431954079\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"
Jul 30 03:45:41.982240 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.982225    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:45:41.988956 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:41.988938    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7lnp\" (UniqueName: \"kubernetes.io/projected/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-kube-api-access-l7lnp\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:45:42.008108 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:42.008090    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-v7mn4"
Jul 30 03:45:42.009047 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:42.009028    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:42.173559 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:42.173515    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-tth9v"]
Jul 30 03:45:42.176688 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:42.176657    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30f6ab6d_dc60_40c5_b6b9_9c22afcaf8e0.slice/crio-98fa0d5baf67074965f4d7f55ac3e65caa0977afbb548e5e96f6e68c5eaa503d WatchSource:0}: Error finding container 98fa0d5baf67074965f4d7f55ac3e65caa0977afbb548e5e96f6e68c5eaa503d: Status 404 returned error can't find the container with id 98fa0d5baf67074965f4d7f55ac3e65caa0977afbb548e5e96f6e68c5eaa503d
Jul 30 03:45:42.191706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:42.191673    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:42.191953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:42.191936    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-v7mn4"]
Jul 30 03:45:42.194720 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:42.194700    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7f4c995_eb94_4d2d_b1c5_d4b375fec8d2.slice/crio-05a7bf87979ba433e7b0eff8f63f2304ab6058736d66b533f09a706de224cc77 WatchSource:0}: Error finding container 05a7bf87979ba433e7b0eff8f63f2304ab6058736d66b533f09a706de224cc77: Status 404 returned error can't find the container with id 05a7bf87979ba433e7b0eff8f63f2304ab6058736d66b533f09a706de224cc77
Jul 30 03:45:42.289317 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:42.289290    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:42.486472 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:42.486414    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tcpmc\" (UID: \"56a7549c-0cff-4485-8518-30d431954079\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"
Jul 30 03:45:42.486599 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:42.486498    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:45:42.486654 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:42.486601    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 03:45:42.486654 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:42.486615    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 03:45:42.486742 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:42.486672    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert podName:56a7549c-0cff-4485-8518-30d431954079 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:43.486652022 +0000 UTC m=+42.525363033 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-tcpmc" (UID: "56a7549c-0cff-4485-8518-30d431954079") : secret "networking-console-plugin-cert" not found
Jul 30 03:45:42.486742 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:42.486699    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls podName:e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:43.486689369 +0000 UTC m=+42.525400372 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ltv96" (UID: "e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7") : secret "cluster-monitoring-operator-tls" not found
Jul 30 03:45:42.707131 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:42.707101    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-v7mn4" event={"ID":"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2","Type":"ContainerStarted","Data":"05a7bf87979ba433e7b0eff8f63f2304ab6058736d66b533f09a706de224cc77"}
Jul 30 03:45:42.708058 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:42.708031    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v" event={"ID":"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0","Type":"ContainerStarted","Data":"98fa0d5baf67074965f4d7f55ac3e65caa0977afbb548e5e96f6e68c5eaa503d"}
Jul 30 03:45:42.918110 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:42.918070    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:43.034856 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:43.034828    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:43.136502 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:43.136467    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:43.385292 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:43.385216    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:43.489905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:43.489870    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:43.494762 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:43.494734    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:45:43.494900 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:43.494788    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tcpmc\" (UID: \"56a7549c-0cff-4485-8518-30d431954079\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"
Jul 30 03:45:43.494957 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:43.494896    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 03:45:43.495017 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:43.494962    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 03:45:43.495017 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:43.494965    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls podName:e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:45.494947629 +0000 UTC m=+44.533658628 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ltv96" (UID: "e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7") : secret "cluster-monitoring-operator-tls" not found
Jul 30 03:45:43.495117 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:43.495025    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert podName:56a7549c-0cff-4485-8518-30d431954079 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:45.495009116 +0000 UTC m=+44.533720119 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-tcpmc" (UID: "56a7549c-0cff-4485-8518-30d431954079") : secret "networking-console-plugin-cert" not found
Jul 30 03:45:43.905826 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:43.905780    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:44.169374 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:44.169287    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:44.225577 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:44.225520    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:44.585497 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:44.585462    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:44.606444 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:44.606415    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:44.606609 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:44.606464    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:44.606680 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:44.606603    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:45:44.606680 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:44.606624    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 03:45:44.606680 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:44.606639    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz: secret "image-registry-tls" not found
Jul 30 03:45:44.606680 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:44.606677    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls podName:9d01562f-999a-4b59-b5fa-7d6738df7633 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:52.606655242 +0000 UTC m=+51.645366260 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls") pod "dns-default-nth9r" (UID: "9d01562f-999a-4b59-b5fa-7d6738df7633") : secret "dns-default-metrics-tls" not found
Jul 30 03:45:44.606828 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:44.606698    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls podName:9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca nodeName:}" failed. No retries permitted until 2026-07-30 03:45:52.606688007 +0000 UTC m=+51.645399010 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls") pod "image-registry-6f9c8fcc6c-hpvwz" (UID: "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca") : secret "image-registry-tls" not found
Jul 30 03:45:44.687513 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:44.687480    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:44.984556 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:44.984470    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:45.040759 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:45.040728    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:45.384688 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:45.384645    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:45.485341 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:45.485306    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:45.514471 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:45.514445    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:45:45.514592 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:45.514505    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tcpmc\" (UID: \"56a7549c-0cff-4485-8518-30d431954079\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"
Jul 30 03:45:45.514665 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:45.514598    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 03:45:45.514665 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:45.514635    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 03:45:45.514665 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:45.514663    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls podName:e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:49.514646352 +0000 UTC m=+48.553357367 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ltv96" (UID: "e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7") : secret "cluster-monitoring-operator-tls" not found
Jul 30 03:45:45.514790 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:45.514697    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert podName:56a7549c-0cff-4485-8518-30d431954079 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:49.51467914 +0000 UTC m=+48.553390139 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-tcpmc" (UID: "56a7549c-0cff-4485-8518-30d431954079") : secret "networking-console-plugin-cert" not found
Jul 30 03:45:45.716882 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:45.716795    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-v7mn4" event={"ID":"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2","Type":"ContainerStarted","Data":"6a3022c9d1c60d0deab9923381f80256eb1577b159460572b316dec260aebc56"}
Jul 30 03:45:45.718310 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:45.718291    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/0.log"
Jul 30 03:45:45.718414 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:45.718325    2569 generic.go:356] "Generic (PLEG): container finished" podID="30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0" containerID="6e8b8f41ebf85198bbb621a9d9adbec86354f1b42cf9faa6a42869f2f340ea9a" exitCode=255
Jul 30 03:45:45.718414 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:45.718352    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v" event={"ID":"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0","Type":"ContainerDied","Data":"6e8b8f41ebf85198bbb621a9d9adbec86354f1b42cf9faa6a42869f2f340ea9a"}
Jul 30 03:45:45.718610 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:45.718595    2569 scope.go:117] "RemoveContainer" containerID="6e8b8f41ebf85198bbb621a9d9adbec86354f1b42cf9faa6a42869f2f340ea9a"
Jul 30 03:45:45.733486 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:45.733449    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-v7mn4" podStartSLOduration=1.622210224 podStartE2EDuration="4.733435676s" totalImagesPullingTime="3.111225452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:45.732477326 +0000 UTC m=+44.771188343" watchObservedRunningTime="2026-07-30 03:45:45.733435676 +0000 UTC m=+44.772146694"
Jul 30 03:45:45.786048 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:45.786021    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:45.886841 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:45.886810    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:46.491205 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.491172    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-88l4r"]
Jul 30 03:45:46.502440 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.502407    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:46.504286 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.504257    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-88l4r"]
Jul 30 03:45:46.504658 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.504637    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 03:45:46.504769 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.504686    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 03:45:46.504769 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.504730    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jpbcm\""
Jul 30 03:45:46.520701 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.520678    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:46.571805 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.571776    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:46.621755 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.621729    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-data-volume\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:46.621831 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.621802    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:46.621876 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.621841    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-crio-socket\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:46.621876 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.621862    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv8z6\" (UniqueName: \"kubernetes.io/projected/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-api-access-qv8z6\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:46.621938 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.621925    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:46.672169 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.672144    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:46.721765 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.721747    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/1.log"
Jul 30 03:45:46.722103 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.722087    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/0.log"
Jul 30 03:45:46.722199 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.722126    2569 generic.go:356] "Generic (PLEG): container finished" podID="30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0" containerID="ccb9bad32167defe5bc03c265db20ab87f2c79f7cb0c8c7d565ca9727f56c3ea" exitCode=255
Jul 30 03:45:46.722247 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.722198    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-data-volume\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:46.722247 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.722214    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v" event={"ID":"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0","Type":"ContainerDied","Data":"ccb9bad32167defe5bc03c265db20ab87f2c79f7cb0c8c7d565ca9727f56c3ea"}
Jul 30 03:45:46.722333 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.722273    2569 scope.go:117] "RemoveContainer" containerID="6e8b8f41ebf85198bbb621a9d9adbec86354f1b42cf9faa6a42869f2f340ea9a"
Jul 30 03:45:46.722333 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.722284    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:46.722421 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.722332    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-crio-socket\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:46.722421 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.722361    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv8z6\" (UniqueName: \"kubernetes.io/projected/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-api-access-qv8z6\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:46.722421 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:46.722406    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 03:45:46.722607 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:46.722478    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 03:45:46.722607 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:46.722483    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm podName:ab25f6a4-deba-4c63-a0a7-3be8b62d67c2 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:47.22245016 +0000 UTC m=+46.261161185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-88l4r" (UID: "ab25f6a4-deba-4c63-a0a7-3be8b62d67c2") : configmap "kube-rbac-proxy" not found
Jul 30 03:45:46.722607 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.722514    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-data-volume\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:46.722607 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.722408    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:46.722607 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:46.722523    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls podName:ab25f6a4-deba-4c63-a0a7-3be8b62d67c2 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:47.222507013 +0000 UTC m=+46.261218025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88l4r" (UID: "ab25f6a4-deba-4c63-a0a7-3be8b62d67c2") : secret "insights-runtime-extractor-tls" not found
Jul 30 03:45:46.722607 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.722565    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-crio-socket\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:46.722923 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.722840    2569 scope.go:117] "RemoveContainer" containerID="ccb9bad32167defe5bc03c265db20ab87f2c79f7cb0c8c7d565ca9727f56c3ea"
Jul 30 03:45:46.723006 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:46.722976    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-tth9v_openshift-console-operator(30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0)\"" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v" podUID="30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0"
Jul 30 03:45:46.723286 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.723261    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:46.730523 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:46.730502    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv8z6\" (UniqueName: \"kubernetes.io/projected/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-api-access-qv8z6\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:47.226725 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:47.226694    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:47.226989 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:47.226789    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:47.226989 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:47.226854    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 03:45:47.226989 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:47.226942    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls podName:ab25f6a4-deba-4c63-a0a7-3be8b62d67c2 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:48.226919846 +0000 UTC m=+47.265630845 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88l4r" (UID: "ab25f6a4-deba-4c63-a0a7-3be8b62d67c2") : secret "insights-runtime-extractor-tls" not found
Jul 30 03:45:47.226989 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:47.226855    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 03:45:47.226989 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:47.226986    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm podName:ab25f6a4-deba-4c63-a0a7-3be8b62d67c2 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:48.226974581 +0000 UTC m=+47.265685579 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-88l4r" (UID: "ab25f6a4-deba-4c63-a0a7-3be8b62d67c2") : configmap "kube-rbac-proxy" not found
Jul 30 03:45:47.726464 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:47.726436    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/1.log"
Jul 30 03:45:47.726898 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:47.726728    2569 scope.go:117] "RemoveContainer" containerID="ccb9bad32167defe5bc03c265db20ab87f2c79f7cb0c8c7d565ca9727f56c3ea"
Jul 30 03:45:47.726898 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:47.726879    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-tth9v_openshift-console-operator(30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0)\"" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v" podUID="30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0"
Jul 30 03:45:48.082401 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:48.082374    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zdwwk_db777952-f0cd-4473-8c10-5ad3c4d1af11/dns-node-resolver/0.log"
Jul 30 03:45:48.234291 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:48.234256    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:48.234451 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:48.234359    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:48.234451 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:48.234403    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 03:45:48.234451 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:48.234447    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 03:45:48.234607 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:48.234467    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls podName:ab25f6a4-deba-4c63-a0a7-3be8b62d67c2 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:50.234452258 +0000 UTC m=+49.273163255 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88l4r" (UID: "ab25f6a4-deba-4c63-a0a7-3be8b62d67c2") : secret "insights-runtime-extractor-tls" not found
Jul 30 03:45:48.234607 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:48.234556    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm podName:ab25f6a4-deba-4c63-a0a7-3be8b62d67c2 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:50.234502559 +0000 UTC m=+49.273213555 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-88l4r" (UID: "ab25f6a4-deba-4c63-a0a7-3be8b62d67c2") : configmap "kube-rbac-proxy" not found
Jul 30 03:45:48.832081 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:48.832049    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:48.881294 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:48.881270    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5dzsm_4b900747-00b2-49fe-8af7-ab67f091664c/node-ca/0.log"
Jul 30 03:45:48.887449 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:48.887422    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:48.977874 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:48.977852    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:49.002224 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.002202    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-m8xwx"]
Jul 30 03:45:49.033916 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.033897    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-m8xwx"]
Jul 30 03:45:49.033996 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.033980    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m8xwx"
Jul 30 03:45:49.035990 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.035970    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 30 03:45:49.036086 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.036048    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 30 03:45:49.036086 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.036078    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-886lj\""
Jul 30 03:45:49.050544 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.050513    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:49.105650 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.105593    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:49.140865 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.140837    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqvl4\" (UniqueName: \"kubernetes.io/projected/a7030d89-984c-4160-a612-fb7431545906-kube-api-access-xqvl4\") pod \"migrator-6865c9967-m8xwx\" (UID: \"a7030d89-984c-4160-a612-fb7431545906\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m8xwx"
Jul 30 03:45:49.241733 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.241709    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqvl4\" (UniqueName: \"kubernetes.io/projected/a7030d89-984c-4160-a612-fb7431545906-kube-api-access-xqvl4\") pod \"migrator-6865c9967-m8xwx\" (UID: \"a7030d89-984c-4160-a612-fb7431545906\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m8xwx"
Jul 30 03:45:49.249175 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.249158    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqvl4\" (UniqueName: \"kubernetes.io/projected/a7030d89-984c-4160-a612-fb7431545906-kube-api-access-xqvl4\") pod \"migrator-6865c9967-m8xwx\" (UID: \"a7030d89-984c-4160-a612-fb7431545906\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m8xwx"
Jul 30 03:45:49.342630 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.342608    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m8xwx"
Jul 30 03:45:49.491671 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.491643    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-m8xwx"]
Jul 30 03:45:49.494602 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:45:49.494577    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7030d89_984c_4160_a612_fb7431545906.slice/crio-bde7c37a89cc8089b1117dd8191a252e7a2d9782e741381ddf7b0d09dff6fe37 WatchSource:0}: Error finding container bde7c37a89cc8089b1117dd8191a252e7a2d9782e741381ddf7b0d09dff6fe37: Status 404 returned error can't find the container with id bde7c37a89cc8089b1117dd8191a252e7a2d9782e741381ddf7b0d09dff6fe37
Jul 30 03:45:49.505672 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.505644    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:49.544740 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.544716    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tcpmc\" (UID: \"56a7549c-0cff-4485-8518-30d431954079\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"
Jul 30 03:45:49.544835 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.544822    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:45:49.544898 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:49.544874    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 03:45:49.544942 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:49.544932    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert podName:56a7549c-0cff-4485-8518-30d431954079 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:57.544916549 +0000 UTC m=+56.583627546 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-tcpmc" (UID: "56a7549c-0cff-4485-8518-30d431954079") : secret "networking-console-plugin-cert" not found
Jul 30 03:45:49.544988 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:49.544946    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 03:45:49.545031 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:49.544988    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls podName:e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:57.544977328 +0000 UTC m=+56.583688324 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ltv96" (UID: "e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7") : secret "cluster-monitoring-operator-tls" not found
Jul 30 03:45:49.610120 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.610093    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:49.731455 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.731395    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m8xwx" event={"ID":"a7030d89-984c-4160-a612-fb7431545906","Type":"ContainerStarted","Data":"bde7c37a89cc8089b1117dd8191a252e7a2d9782e741381ddf7b0d09dff6fe37"}
Jul 30 03:45:49.881489 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:49.881460    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lzf7c_ad3a8ea4-d784-43ae-832c-16a6ebf99c1f/serve-healthcheck-canary/0.log"
Jul 30 03:45:50.251266 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:50.251232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:50.251413 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:50.251292    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:50.251413 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:50.251359    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 03:45:50.251413 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:50.251374    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 03:45:50.251518 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:50.251419    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls podName:ab25f6a4-deba-4c63-a0a7-3be8b62d67c2 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:54.251405274 +0000 UTC m=+53.290116271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88l4r" (UID: "ab25f6a4-deba-4c63-a0a7-3be8b62d67c2") : secret "insights-runtime-extractor-tls" not found
Jul 30 03:45:50.251518 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:50.251431    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm podName:ab25f6a4-deba-4c63-a0a7-3be8b62d67c2 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:54.251425442 +0000 UTC m=+53.290136439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-88l4r" (UID: "ab25f6a4-deba-4c63-a0a7-3be8b62d67c2") : configmap "kube-rbac-proxy" not found
Jul 30 03:45:51.131486 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:51.131455    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:51.737573 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:51.737482    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m8xwx" event={"ID":"a7030d89-984c-4160-a612-fb7431545906","Type":"ContainerStarted","Data":"8f601028e81cabbc933075a414ae4c0318d27a1aaf555ff98e633bf6a36274ee"}
Jul 30 03:45:51.737573 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:51.737541    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m8xwx" event={"ID":"a7030d89-984c-4160-a612-fb7431545906","Type":"ContainerStarted","Data":"330fcc74b09fc68bc6d47621615c90dba571ec344ef905a659878d6b9ac1da3c"}
Jul 30 03:45:51.752914 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:51.752834    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m8xwx" podStartSLOduration=2.057656146 podStartE2EDuration="3.75281947s" totalImagesPullingTime="1.695163324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:51.751789377 +0000 UTC m=+50.790500400" watchObservedRunningTime="2026-07-30 03:45:51.75281947 +0000 UTC m=+50.791530489"
Jul 30 03:45:51.936251 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:51.936222    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:51.981432 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:51.981406    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:51.981578 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:51.981462    2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:45:51.981828 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:51.981812    2569 scope.go:117] "RemoveContainer" containerID="ccb9bad32167defe5bc03c265db20ab87f2c79f7cb0c8c7d565ca9727f56c3ea"
Jul 30 03:45:51.981992 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:51.981972    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-tth9v_openshift-console-operator(30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0)\"" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v" podUID="30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0"
Jul 30 03:45:52.671073 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:52.671039    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:45:52.671472 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:52.671088    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:45:52.671472 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:52.671169    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:45:52.671472 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:52.671221    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls podName:9d01562f-999a-4b59-b5fa-7d6738df7633 nodeName:}" failed. No retries permitted until 2026-07-30 03:46:08.671207538 +0000 UTC m=+67.709918535 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls") pod "dns-default-nth9r" (UID: "9d01562f-999a-4b59-b5fa-7d6738df7633") : secret "dns-default-metrics-tls" not found
Jul 30 03:45:52.671472 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:52.671232    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 03:45:52.671472 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:52.671247    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz: secret "image-registry-tls" not found
Jul 30 03:45:52.671472 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:52.671286    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls podName:9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca nodeName:}" failed. No retries permitted until 2026-07-30 03:46:08.671274204 +0000 UTC m=+67.709985204 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls") pod "image-registry-6f9c8fcc6c-hpvwz" (UID: "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca") : secret "image-registry-tls" not found
Jul 30 03:45:53.061837 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:53.061808    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:53.274212 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:53.274181    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:53.375328 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:53.375261    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:53.474624 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:53.474592    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:53.855025 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:53.854990    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:54.157253 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:54.157169    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:54.283409 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:54.283382    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:54.283564 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:54.283481    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:54.283564 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:54.283548    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 03:45:54.283641 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:54.283615    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls podName:ab25f6a4-deba-4c63-a0a7-3be8b62d67c2 nodeName:}" failed. No retries permitted until 2026-07-30 03:46:02.283600663 +0000 UTC m=+61.322311662 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88l4r" (UID: "ab25f6a4-deba-4c63-a0a7-3be8b62d67c2") : secret "insights-runtime-extractor-tls" not found
Jul 30 03:45:54.283969 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:54.283953    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:45:54.347822 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:54.347793    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:54.501128 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:54.501059    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:54.602193 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:54.602168    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:55.544743 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:55.544703    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:55.755795 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:55.755765    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:56.146220 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:56.146184    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:56.378881 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:56.378851    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:56.745154 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:56.745110    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:56.795347 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:56.795318    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:57.049560 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:57.049516    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:57.167109 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:57.167085    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:57.549748 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:57.549721    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:57.598701 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:57.598680    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:57.606592 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:57.606569    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:45:57.606679 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:57.606616    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tcpmc\" (UID: \"56a7549c-0cff-4485-8518-30d431954079\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"
Jul 30 03:45:57.606716 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:57.606709    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 03:45:57.606748 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:57.606709    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 03:45:57.606780 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:57.606760    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert podName:56a7549c-0cff-4485-8518-30d431954079 nodeName:}" failed. No retries permitted until 2026-07-30 03:46:13.606744846 +0000 UTC m=+72.645455860 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-tcpmc" (UID: "56a7549c-0cff-4485-8518-30d431954079") : secret "networking-console-plugin-cert" not found
Jul 30 03:45:57.606780 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:45:57.606773    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls podName:e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7 nodeName:}" failed. No retries permitted until 2026-07-30 03:46:13.60676745 +0000 UTC m=+72.645478448 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ltv96" (UID: "e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7") : secret "cluster-monitoring-operator-tls" not found
Jul 30 03:45:58.152898 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:58.152863    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:58.557063 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:58.557029    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:58.867607 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:58.867514    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:58.945911 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:58.945879    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:59.271248 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:59.271218    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:59.370822 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:59.370797    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:59.547001 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:59.546921    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:59.646665 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:59.646632    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:59.747398 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:45:59.747368    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:00.144442 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:00.144406    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:00.548170 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:00.548139    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:00.942830 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:00.942803    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:01.348635 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:01.348602    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:01.545900 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:01.545870    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:01.671439 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:01.671354    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:01.771744 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:01.771711    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:01.966490 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:01.966412    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:02.342861 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:02.342820    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:46:02.343011 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:46:02.342987    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 03:46:02.343061 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:46:02.343051    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls podName:ab25f6a4-deba-4c63-a0a7-3be8b62d67c2 nodeName:}" failed. No retries permitted until 2026-07-30 03:46:18.343035277 +0000 UTC m=+77.381746275 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88l4r" (UID: "ab25f6a4-deba-4c63-a0a7-3be8b62d67c2") : secret "insights-runtime-extractor-tls" not found
Jul 30 03:46:02.543283 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:02.543255    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:02.681017 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:02.680963    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6887b"
Jul 30 03:46:03.149545 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:03.149494    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:03.348135 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:03.348095    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:03.544185 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:03.544153    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:03.746476 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:03.746448    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:04.460279 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:04.460254    2569 scope.go:117] "RemoveContainer" containerID="ccb9bad32167defe5bc03c265db20ab87f2c79f7cb0c8c7d565ca9727f56c3ea"
Jul 30 03:46:04.771798 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:04.771775    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 03:46:04.772162 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:04.772140    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/1.log"
Jul 30 03:46:04.772219 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:04.772175    2569 generic.go:356] "Generic (PLEG): container finished" podID="30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0" containerID="1914330f900c5a5b49ff6a4d7687fb24bfb3fa9b9bbde4beff9eb1d53f477fef" exitCode=255
Jul 30 03:46:04.772253 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:04.772217    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v" event={"ID":"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0","Type":"ContainerDied","Data":"1914330f900c5a5b49ff6a4d7687fb24bfb3fa9b9bbde4beff9eb1d53f477fef"}
Jul 30 03:46:04.772302 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:04.772267    2569 scope.go:117] "RemoveContainer" containerID="ccb9bad32167defe5bc03c265db20ab87f2c79f7cb0c8c7d565ca9727f56c3ea"
Jul 30 03:46:04.772549 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:04.772514    2569 scope.go:117] "RemoveContainer" containerID="1914330f900c5a5b49ff6a4d7687fb24bfb3fa9b9bbde4beff9eb1d53f477fef"
Jul 30 03:46:04.772725 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:46:04.772707    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-tth9v_openshift-console-operator(30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0)\"" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v" podUID="30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0"
Jul 30 03:46:05.775575 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:05.775552    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 03:46:07.283337 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.283295    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs\") pod \"network-metrics-daemon-nsf2h\" (UID: \"30c27db8-8cfe-492d-855d-4e085771ed3e\") " pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:46:07.283337 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.283344    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:46:07.285761 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.285743    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 03:46:07.286425 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.286410    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 03:46:07.296126 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.296097    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26002ce-a680-4af4-9513-044a00d2393d-original-pull-secret\") pod \"global-pull-secret-syncer-vxddq\" (UID: \"a26002ce-a680-4af4-9513-044a00d2393d\") " pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:46:07.296226 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.296097    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30c27db8-8cfe-492d-855d-4e085771ed3e-metrics-certs\") pod \"network-metrics-daemon-nsf2h\" (UID: \"30c27db8-8cfe-492d-855d-4e085771ed3e\") " pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:46:07.384084 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.384047    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cmgt\" (UniqueName: \"kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt\") pod \"network-check-target-bpwlj\" (UID: \"0295adc4-b5c7-42c2-ae4c-b653ae27ca5c\") " pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:46:07.386279 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.386259    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 03:46:07.396567 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.396547    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 03:46:07.407539 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.407513    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cmgt\" (UniqueName: \"kubernetes.io/projected/0295adc4-b5c7-42c2-ae4c-b653ae27ca5c-kube-api-access-5cmgt\") pod \"network-check-target-bpwlj\" (UID: \"0295adc4-b5c7-42c2-ae4c-b653ae27ca5c\") " pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:46:07.471026 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.471001    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxddq"
Jul 30 03:46:07.478964 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.478943    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hmgxg\""
Jul 30 03:46:07.486854 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.486836    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kzw94\""
Jul 30 03:46:07.486949 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.486842    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:46:07.495233 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.495210    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nsf2h"
Jul 30 03:46:07.613997 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.613919    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vxddq"]
Jul 30 03:46:07.618133 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:46:07.618108    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda26002ce_a680_4af4_9513_044a00d2393d.slice/crio-b5957274c45fb3af2ab18ca5ed7321df45bb4daa439af45bdf3a7678c896bf0d WatchSource:0}: Error finding container b5957274c45fb3af2ab18ca5ed7321df45bb4daa439af45bdf3a7678c896bf0d: Status 404 returned error can't find the container with id b5957274c45fb3af2ab18ca5ed7321df45bb4daa439af45bdf3a7678c896bf0d
Jul 30 03:46:07.632006 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.631983    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bpwlj"]
Jul 30 03:46:07.634894 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:46:07.634865    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0295adc4_b5c7_42c2_ae4c_b653ae27ca5c.slice/crio-24ef774cb7f413d10d4c0e44b76590efff691e13bfe8e51e1e702f2b806024c3 WatchSource:0}: Error finding container 24ef774cb7f413d10d4c0e44b76590efff691e13bfe8e51e1e702f2b806024c3: Status 404 returned error can't find the container with id 24ef774cb7f413d10d4c0e44b76590efff691e13bfe8e51e1e702f2b806024c3
Jul 30 03:46:07.646939 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.646913    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nsf2h"]
Jul 30 03:46:07.651737 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:46:07.651714    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30c27db8_8cfe_492d_855d_4e085771ed3e.slice/crio-bc5c4874f61f475d140e6ab37ef4dd231155fef46a57e09e13cdd91e590da56a WatchSource:0}: Error finding container bc5c4874f61f475d140e6ab37ef4dd231155fef46a57e09e13cdd91e590da56a: Status 404 returned error can't find the container with id bc5c4874f61f475d140e6ab37ef4dd231155fef46a57e09e13cdd91e590da56a
Jul 30 03:46:07.781060 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.781025    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bpwlj" event={"ID":"0295adc4-b5c7-42c2-ae4c-b653ae27ca5c","Type":"ContainerStarted","Data":"24ef774cb7f413d10d4c0e44b76590efff691e13bfe8e51e1e702f2b806024c3"}
Jul 30 03:46:07.782043 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.782021    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vxddq" event={"ID":"a26002ce-a680-4af4-9513-044a00d2393d","Type":"ContainerStarted","Data":"b5957274c45fb3af2ab18ca5ed7321df45bb4daa439af45bdf3a7678c896bf0d"}
Jul 30 03:46:07.782968 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:07.782942    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nsf2h" event={"ID":"30c27db8-8cfe-492d-855d-4e085771ed3e","Type":"ContainerStarted","Data":"bc5c4874f61f475d140e6ab37ef4dd231155fef46a57e09e13cdd91e590da56a"}
Jul 30 03:46:08.697232 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:08.697198    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:46:08.697800 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:08.697298    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:46:08.699934 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:08.699903    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls\") pod \"image-registry-6f9c8fcc6c-hpvwz\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") " pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:46:08.700049 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:08.699969    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d01562f-999a-4b59-b5fa-7d6738df7633-metrics-tls\") pod \"dns-default-nth9r\" (UID: \"9d01562f-999a-4b59-b5fa-7d6738df7633\") " pod="openshift-dns/dns-default-nth9r"
Jul 30 03:46:08.912787 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:08.912748    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tfs77\""
Jul 30 03:46:08.921387 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:08.921367    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:46:08.931930 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:08.931911    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5s29r\""
Jul 30 03:46:08.939170 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:08.939145    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nth9r"
Jul 30 03:46:09.082819 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.082788    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"]
Jul 30 03:46:09.087784 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:46:09.087756    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b0bf04d_3aa6_416c_9b06_cc82ea8fb2ca.slice/crio-e166be2460b1cea12764e7b8b7496d529d24c41b4c99776fbdf532c6398cd385 WatchSource:0}: Error finding container e166be2460b1cea12764e7b8b7496d529d24c41b4c99776fbdf532c6398cd385: Status 404 returned error can't find the container with id e166be2460b1cea12764e7b8b7496d529d24c41b4c99776fbdf532c6398cd385
Jul 30 03:46:09.099428 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.099400    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nth9r"]
Jul 30 03:46:09.103081 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:46:09.103059    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d01562f_999a_4b59_b5fa_7d6738df7633.slice/crio-7455cbba2f0e7bfe1315f2897574d4d3488d5ac8acde371e69b8704d91ab15f3 WatchSource:0}: Error finding container 7455cbba2f0e7bfe1315f2897574d4d3488d5ac8acde371e69b8704d91ab15f3: Status 404 returned error can't find the container with id 7455cbba2f0e7bfe1315f2897574d4d3488d5ac8acde371e69b8704d91ab15f3
Jul 30 03:46:09.791420 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.790882    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz" event={"ID":"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca","Type":"ContainerStarted","Data":"3fe6585c853bd918876cc4a98739203f67f31c0787f50d7def7f58bbb16d9f77"}
Jul 30 03:46:09.791420 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.790941    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz" event={"ID":"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca","Type":"ContainerStarted","Data":"e166be2460b1cea12764e7b8b7496d529d24c41b4c99776fbdf532c6398cd385"}
Jul 30 03:46:09.791420 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.790971    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:46:09.792733 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.792700    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nth9r" event={"ID":"9d01562f-999a-4b59-b5fa-7d6738df7633","Type":"ContainerStarted","Data":"7455cbba2f0e7bfe1315f2897574d4d3488d5ac8acde371e69b8704d91ab15f3"}
Jul 30 03:46:09.794855 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.794831    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nsf2h" event={"ID":"30c27db8-8cfe-492d-855d-4e085771ed3e","Type":"ContainerStarted","Data":"00719b34ae769c688f0ad8860565c66c79eed1a977c3e95b2a51be47be3c2168"}
Jul 30 03:46:09.794972 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.794879    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nsf2h" event={"ID":"30c27db8-8cfe-492d-855d-4e085771ed3e","Type":"ContainerStarted","Data":"23913e4b198d1a638676551a6d333d65c303393e3faeab102662d4506634d455"}
Jul 30 03:46:09.816065 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.816015    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz" podStartSLOduration=45.815998461 podStartE2EDuration="45.815998461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:09.815117614 +0000 UTC m=+68.853828632" watchObservedRunningTime="2026-07-30 03:46:09.815998461 +0000 UTC m=+68.854709477"
Jul 30 03:46:09.830900 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.830851    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c9c5b44cd-h4c66"]
Jul 30 03:46:09.837457 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.837411    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nsf2h" podStartSLOduration=66.759624992 podStartE2EDuration="1m7.837398275s" totalImagesPullingTime="1.077773283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:09.836244379 +0000 UTC m=+68.874955394" watchObservedRunningTime="2026-07-30 03:46:09.837398275 +0000 UTC m=+68.876109294"
Jul 30 03:46:09.848931 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.848907    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"]
Jul 30 03:46:09.849804 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.849769    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c9c5b44cd-h4c66"
Jul 30 03:46:09.852077 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.852052    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 30 03:46:09.853171 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.852844    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 30 03:46:09.853171 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.852863    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 30 03:46:09.853171 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.853046    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-t929j\""
Jul 30 03:46:09.853692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.853419    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 30 03:46:09.865156 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.865130    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c9c5b44cd-h4c66"]
Jul 30 03:46:09.865272 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.865160    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"]
Jul 30 03:46:09.865272 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.865250    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"
Jul 30 03:46:09.867328 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.867313    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 30 03:46:09.907921 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.907753    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:09.908514 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.908430    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad07a69f-3f1a-4b4f-b680-23fcc367f707-tmp\") pod \"klusterlet-addon-workmgr-644df444d6-b6tq2\" (UID: \"ad07a69f-3f1a-4b4f-b680-23fcc367f707\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"
Jul 30 03:46:09.908653 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.908552    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x58dj\" (UniqueName: \"kubernetes.io/projected/ad07a69f-3f1a-4b4f-b680-23fcc367f707-kube-api-access-x58dj\") pod \"klusterlet-addon-workmgr-644df444d6-b6tq2\" (UID: \"ad07a69f-3f1a-4b4f-b680-23fcc367f707\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"
Jul 30 03:46:09.908843 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.908819    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75scr\" (UniqueName: \"kubernetes.io/projected/0ed6ef8c-04f8-4852-a3dc-ff45d29a3f51-kube-api-access-75scr\") pod \"managed-serviceaccount-addon-agent-c9c5b44cd-h4c66\" (UID: \"0ed6ef8c-04f8-4852-a3dc-ff45d29a3f51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c9c5b44cd-h4c66"
Jul 30 03:46:09.909272 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.908966    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0ed6ef8c-04f8-4852-a3dc-ff45d29a3f51-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-c9c5b44cd-h4c66\" (UID: \"0ed6ef8c-04f8-4852-a3dc-ff45d29a3f51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c9c5b44cd-h4c66"
Jul 30 03:46:09.909384 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:09.909235    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ad07a69f-3f1a-4b4f-b680-23fcc367f707-klusterlet-config\") pod \"klusterlet-addon-workmgr-644df444d6-b6tq2\" (UID: \"ad07a69f-3f1a-4b4f-b680-23fcc367f707\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"
Jul 30 03:46:10.010117 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.010080    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0ed6ef8c-04f8-4852-a3dc-ff45d29a3f51-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-c9c5b44cd-h4c66\" (UID: \"0ed6ef8c-04f8-4852-a3dc-ff45d29a3f51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c9c5b44cd-h4c66"
Jul 30 03:46:10.010308 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.010162    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ad07a69f-3f1a-4b4f-b680-23fcc367f707-klusterlet-config\") pod \"klusterlet-addon-workmgr-644df444d6-b6tq2\" (UID: \"ad07a69f-3f1a-4b4f-b680-23fcc367f707\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"
Jul 30 03:46:10.010308 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.010254    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad07a69f-3f1a-4b4f-b680-23fcc367f707-tmp\") pod \"klusterlet-addon-workmgr-644df444d6-b6tq2\" (UID: \"ad07a69f-3f1a-4b4f-b680-23fcc367f707\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"
Jul 30 03:46:10.010308 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.010283    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x58dj\" (UniqueName: \"kubernetes.io/projected/ad07a69f-3f1a-4b4f-b680-23fcc367f707-kube-api-access-x58dj\") pod \"klusterlet-addon-workmgr-644df444d6-b6tq2\" (UID: \"ad07a69f-3f1a-4b4f-b680-23fcc367f707\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"
Jul 30 03:46:10.010476 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.010316    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75scr\" (UniqueName: \"kubernetes.io/projected/0ed6ef8c-04f8-4852-a3dc-ff45d29a3f51-kube-api-access-75scr\") pod \"managed-serviceaccount-addon-agent-c9c5b44cd-h4c66\" (UID: \"0ed6ef8c-04f8-4852-a3dc-ff45d29a3f51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c9c5b44cd-h4c66"
Jul 30 03:46:10.010663 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.010636    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad07a69f-3f1a-4b4f-b680-23fcc367f707-tmp\") pod \"klusterlet-addon-workmgr-644df444d6-b6tq2\" (UID: \"ad07a69f-3f1a-4b4f-b680-23fcc367f707\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"
Jul 30 03:46:10.013409 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.013382    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ad07a69f-3f1a-4b4f-b680-23fcc367f707-klusterlet-config\") pod \"klusterlet-addon-workmgr-644df444d6-b6tq2\" (UID: \"ad07a69f-3f1a-4b4f-b680-23fcc367f707\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"
Jul 30 03:46:10.016254 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.016212    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0ed6ef8c-04f8-4852-a3dc-ff45d29a3f51-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-c9c5b44cd-h4c66\" (UID: \"0ed6ef8c-04f8-4852-a3dc-ff45d29a3f51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c9c5b44cd-h4c66"
Jul 30 03:46:10.029298 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.029273    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x58dj\" (UniqueName: \"kubernetes.io/projected/ad07a69f-3f1a-4b4f-b680-23fcc367f707-kube-api-access-x58dj\") pod \"klusterlet-addon-workmgr-644df444d6-b6tq2\" (UID: \"ad07a69f-3f1a-4b4f-b680-23fcc367f707\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"
Jul 30 03:46:10.033007 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.032964    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75scr\" (UniqueName: \"kubernetes.io/projected/0ed6ef8c-04f8-4852-a3dc-ff45d29a3f51-kube-api-access-75scr\") pod \"managed-serviceaccount-addon-agent-c9c5b44cd-h4c66\" (UID: \"0ed6ef8c-04f8-4852-a3dc-ff45d29a3f51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c9c5b44cd-h4c66"
Jul 30 03:46:10.058248 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.058187    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:10.159420 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.159382    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:10.173890 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.173645    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c9c5b44cd-h4c66"
Jul 30 03:46:10.183320 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.183289    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"
Jul 30 03:46:10.256585 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.256526    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:10.487628 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:10.487508    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:11.185454 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:11.185425    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"]
Jul 30 03:46:11.205004 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:11.204974    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c9c5b44cd-h4c66"]
Jul 30 03:46:11.334994 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:46:11.334923    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad07a69f_3f1a_4b4f_b680_23fcc367f707.slice/crio-9ed8e5f20e36388d6c03336568fc0d16fec0a53e0349bd2c903de7dcf046c516 WatchSource:0}: Error finding container 9ed8e5f20e36388d6c03336568fc0d16fec0a53e0349bd2c903de7dcf046c516: Status 404 returned error can't find the container with id 9ed8e5f20e36388d6c03336568fc0d16fec0a53e0349bd2c903de7dcf046c516
Jul 30 03:46:11.335960 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:46:11.335923    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ed6ef8c_04f8_4852_a3dc_ff45d29a3f51.slice/crio-7efe783a10012b623d5708bdef269ac684adf0017bb810a8e628488d9df2b71a WatchSource:0}: Error finding container 7efe783a10012b623d5708bdef269ac684adf0017bb810a8e628488d9df2b71a: Status 404 returned error can't find the container with id 7efe783a10012b623d5708bdef269ac684adf0017bb810a8e628488d9df2b71a
Jul 30 03:46:11.804381 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:11.804335    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c9c5b44cd-h4c66" event={"ID":"0ed6ef8c-04f8-4852-a3dc-ff45d29a3f51","Type":"ContainerStarted","Data":"7efe783a10012b623d5708bdef269ac684adf0017bb810a8e628488d9df2b71a"}
Jul 30 03:46:11.805718 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:11.805680    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2" event={"ID":"ad07a69f-3f1a-4b4f-b680-23fcc367f707","Type":"ContainerStarted","Data":"9ed8e5f20e36388d6c03336568fc0d16fec0a53e0349bd2c903de7dcf046c516"}
Jul 30 03:46:11.982049 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:11.982013    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:46:11.982193 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:11.982076    2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:46:11.982431 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:11.982410    2569 scope.go:117] "RemoveContainer" containerID="1914330f900c5a5b49ff6a4d7687fb24bfb3fa9b9bbde4beff9eb1d53f477fef"
Jul 30 03:46:11.982626 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:46:11.982604    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-tth9v_openshift-console-operator(30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0)\"" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v" podUID="30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0"
Jul 30 03:46:12.812191 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:12.812146    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bpwlj" event={"ID":"0295adc4-b5c7-42c2-ae4c-b653ae27ca5c","Type":"ContainerStarted","Data":"c6af0c4c98a3cf8900da90f5f623f795b0abaecb81a3cfda95254cade3bd99c0"}
Jul 30 03:46:12.812705 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:12.812608    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:46:12.818022 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:12.817263    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nth9r" event={"ID":"9d01562f-999a-4b59-b5fa-7d6738df7633","Type":"ContainerStarted","Data":"6206355b981f0ed53cd84ed1e582163b0a3998deb5b59d94b70490d7143d0624"}
Jul 30 03:46:12.818022 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:12.817311    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nth9r" event={"ID":"9d01562f-999a-4b59-b5fa-7d6738df7633","Type":"ContainerStarted","Data":"f7dc3e8661137281f173e85bf35dd0678a20de75cfcc312f923404f9a87cabe8"}
Jul 30 03:46:12.818022 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:12.817828    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nth9r"
Jul 30 03:46:12.845760 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:12.845365    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bpwlj" podStartSLOduration=66.514429562 podStartE2EDuration="1m10.845345262s" totalImagesPullingTime="4.3309157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:12.828372475 +0000 UTC m=+71.867083494" watchObservedRunningTime="2026-07-30 03:46:12.845345262 +0000 UTC m=+71.884056282"
Jul 30 03:46:13.647479 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:13.647442    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:46:13.647672 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:13.647517    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tcpmc\" (UID: \"56a7549c-0cff-4485-8518-30d431954079\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"
Jul 30 03:46:13.650508 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:13.650478    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56a7549c-0cff-4485-8518-30d431954079-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tcpmc\" (UID: \"56a7549c-0cff-4485-8518-30d431954079\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"
Jul 30 03:46:13.651767 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:13.651744    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ltv96\" (UID: \"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:46:13.862305 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:13.862096    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-c8qpq\""
Jul 30 03:46:13.870668 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:13.870643    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"
Jul 30 03:46:13.872285 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:13.872263    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tqpmr\""
Jul 30 03:46:13.881736 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:13.881714    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"
Jul 30 03:46:16.348935 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.348896    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:16.409888 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.409850    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:16.675680 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.675513    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:16.729792 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.729408    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:16.798125 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.797815    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nth9r" podStartSLOduration=37.742258847 podStartE2EDuration="40.797796131s" totalImagesPullingTime="3.055537284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:12.844908276 +0000 UTC m=+71.883619297" watchObservedRunningTime="2026-07-30 03:46:16.797796131 +0000 UTC m=+75.836507149"
Jul 30 03:46:16.798125 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.797997    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96"]
Jul 30 03:46:16.802398 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:46:16.802363    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6c5fbeb_b997_4d55_b2bc_9eeed9a36fa7.slice/crio-f8c0b30982a337a7fc3b517bedb2ae76acd26956513d25bdf2cbd36cbd636f1b WatchSource:0}: Error finding container f8c0b30982a337a7fc3b517bedb2ae76acd26956513d25bdf2cbd36cbd636f1b: Status 404 returned error can't find the container with id f8c0b30982a337a7fc3b517bedb2ae76acd26956513d25bdf2cbd36cbd636f1b
Jul 30 03:46:16.824760 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.824726    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc"]
Jul 30 03:46:16.828169 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:46:16.828147    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a7549c_0cff_4485_8518_30d431954079.slice/crio-7fc1c690cfdf9c089627b27398874da826edde0f6db56897718ad7c0c9b3e497 WatchSource:0}: Error finding container 7fc1c690cfdf9c089627b27398874da826edde0f6db56897718ad7c0c9b3e497: Status 404 returned error can't find the container with id 7fc1c690cfdf9c089627b27398874da826edde0f6db56897718ad7c0c9b3e497
Jul 30 03:46:16.835044 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.834816    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vxddq" event={"ID":"a26002ce-a680-4af4-9513-044a00d2393d","Type":"ContainerStarted","Data":"f151878cb286f7fc0795b1d7f428127994900157bbca64975edeffb4f3ddcabf"}
Jul 30 03:46:16.836233 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.836209    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c9c5b44cd-h4c66" event={"ID":"0ed6ef8c-04f8-4852-a3dc-ff45d29a3f51","Type":"ContainerStarted","Data":"ca1de369e1dc8bdb040d46e4ab19d60003fd692ad39931a4f0523482b52adfd8"}
Jul 30 03:46:16.838417 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.838391    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2" event={"ID":"ad07a69f-3f1a-4b4f-b680-23fcc367f707","Type":"ContainerStarted","Data":"0b1c5829db2eb74752ffb05cd2b9a25453b51b151f0db4eeb3428a48d77c585b"}
Jul 30 03:46:16.838739 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.838623    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"
Jul 30 03:46:16.840046 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.840008    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2"
Jul 30 03:46:16.840275 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.840255    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc" event={"ID":"56a7549c-0cff-4485-8518-30d431954079","Type":"ContainerStarted","Data":"7fc1c690cfdf9c089627b27398874da826edde0f6db56897718ad7c0c9b3e497"}
Jul 30 03:46:16.841634 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.841581    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96" event={"ID":"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7","Type":"ContainerStarted","Data":"f8c0b30982a337a7fc3b517bedb2ae76acd26956513d25bdf2cbd36cbd636f1b"}
Jul 30 03:46:16.847893 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.847854    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vxddq" podStartSLOduration=65.84526382 podStartE2EDuration="1m14.84784233s" totalImagesPullingTime="9.00257851s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:16.84692749 +0000 UTC m=+75.885638510" watchObservedRunningTime="2026-07-30 03:46:16.84784233 +0000 UTC m=+75.886553326"
Jul 30 03:46:16.860297 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.860230    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c9c5b44cd-h4c66" podStartSLOduration=2.577024445 podStartE2EDuration="7.860179436s" totalImagesPullingTime="5.283154991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:16.859746615 +0000 UTC m=+75.898457633" watchObservedRunningTime="2026-07-30 03:46:16.860179436 +0000 UTC m=+75.898890446"
Jul 30 03:46:16.877458 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:16.877368    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-644df444d6-b6tq2" podStartSLOduration=2.583744039 podStartE2EDuration="7.877351213s" totalImagesPullingTime="5.293607174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:16.876818963 +0000 UTC m=+75.915529984" watchObservedRunningTime="2026-07-30 03:46:16.877351213 +0000 UTC m=+75.916062245"
Jul 30 03:46:18.388513 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:18.388475    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:46:18.391144 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:18.391106    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab25f6a4-deba-4c63-a0a7-3be8b62d67c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88l4r\" (UID: \"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2\") " pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:46:18.613890 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:18.613856    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jpbcm\""
Jul 30 03:46:18.622198 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:18.622177    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-88l4r"
Jul 30 03:46:18.755440 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:18.755406    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-88l4r"]
Jul 30 03:46:18.758730 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:46:18.758699    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab25f6a4_deba_4c63_a0a7_3be8b62d67c2.slice/crio-9d3f096462dd8d80ecf9b7d70a2837d1c38bfc960e53e44daf2b661f890b3b86 WatchSource:0}: Error finding container 9d3f096462dd8d80ecf9b7d70a2837d1c38bfc960e53e44daf2b661f890b3b86: Status 404 returned error can't find the container with id 9d3f096462dd8d80ecf9b7d70a2837d1c38bfc960e53e44daf2b661f890b3b86
Jul 30 03:46:18.850442 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:18.850407    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc" event={"ID":"56a7549c-0cff-4485-8518-30d431954079","Type":"ContainerStarted","Data":"2ed9602df3824f055066d5552d3c06ba691b125ea79ba246c6b7f3ba9bb748aa"}
Jul 30 03:46:18.852081 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:18.852055    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96" event={"ID":"e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7","Type":"ContainerStarted","Data":"cfcd9bdb08cef327788938d62c9b1f800ca6992b6951e3e252ca383c135b760c"}
Jul 30 03:46:18.853161 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:18.853142    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-88l4r" event={"ID":"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2","Type":"ContainerStarted","Data":"9d3f096462dd8d80ecf9b7d70a2837d1c38bfc960e53e44daf2b661f890b3b86"}
Jul 30 03:46:18.864184 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:18.864142    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-tcpmc" podStartSLOduration=36.074128473 podStartE2EDuration="37.864130166s" totalImagesPullingTime="1.790001693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:18.863939729 +0000 UTC m=+77.902650748" watchObservedRunningTime="2026-07-30 03:46:18.864130166 +0000 UTC m=+77.902841176"
Jul 30 03:46:18.879131 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:18.879090    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ltv96" podStartSLOduration=36.057231149 podStartE2EDuration="37.879077723s" totalImagesPullingTime="1.821846574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:18.877742927 +0000 UTC m=+77.916453950" watchObservedRunningTime="2026-07-30 03:46:18.879077723 +0000 UTC m=+77.917788743"
Jul 30 03:46:18.996621 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:18.996527    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:19.052658 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:19.052625    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:19.127374 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:19.127340    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:20.419069 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:20.419016    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:20.994269 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:20.994235    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:21.047116 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:21.047071    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:21.120164 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:21.120122    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:23.162943 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:23.162807    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:23.223006 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:23.222972    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:23.460165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:23.460097    2569 scope.go:117] "RemoveContainer" containerID="1914330f900c5a5b49ff6a4d7687fb24bfb3fa9b9bbde4beff9eb1d53f477fef"
Jul 30 03:46:23.460320 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:46:23.460298    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-tth9v_openshift-console-operator(30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0)\"" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v" podUID="30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0"
Jul 30 03:46:23.482202 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:23.482164    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:23.826849 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:23.826819    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nth9r"
Jul 30 03:46:23.874445 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:23.874414    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-88l4r" event={"ID":"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2","Type":"ContainerStarted","Data":"1d7e4951e14d8d86c2bf2da9d8e7b991be1e9a03b51b8018a659d69ff68be0a0"}
Jul 30 03:46:26.985031 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:26.984996    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:27.891414 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:27.891381    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-88l4r" event={"ID":"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2","Type":"ContainerStarted","Data":"8217aa6c2b53e0426e3d1f0c249191cd844ccb0bc1b85b6f17360918c9bed54d"}
Jul 30 03:46:28.122072 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.122032    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:28.180242 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.180138    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:28.300026 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.299943    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:28.513248 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.513157    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:28.572245 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.572207    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qd6jz"]
Jul 30 03:46:28.575635 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.575609    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.577770 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.577699    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 03:46:28.577916 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.577847    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fg6jg\""
Jul 30 03:46:28.577916 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.577871    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 03:46:28.578882 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.578860    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 03:46:28.579109 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.579093    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 03:46:28.598878 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.598845    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:28.659231 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.659202    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:28.671148 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.671122    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-accelerators-collector-config\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.671286 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.671165    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04b2f1f3-f80c-4137-973d-e789d2c488ce-root\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.671286 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.671192    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04b2f1f3-f80c-4137-973d-e789d2c488ce-metrics-client-ca\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.671286 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.671218    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-wtmp\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.671286 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.671243    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.671286 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.671268    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f7vk\" (UniqueName: \"kubernetes.io/projected/04b2f1f3-f80c-4137-973d-e789d2c488ce-kube-api-access-6f7vk\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.671554 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.671381    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-tls\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.671554 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.671423    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04b2f1f3-f80c-4137-973d-e789d2c488ce-sys\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.671554 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.671449    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-textfile\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.772429 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.772398    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-accelerators-collector-config\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.772611 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.772459    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04b2f1f3-f80c-4137-973d-e789d2c488ce-root\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.772611 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.772485    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04b2f1f3-f80c-4137-973d-e789d2c488ce-metrics-client-ca\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.772611 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.772510    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-wtmp\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.772611 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.772553    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.772611 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.772577    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f7vk\" (UniqueName: \"kubernetes.io/projected/04b2f1f3-f80c-4137-973d-e789d2c488ce-kube-api-access-6f7vk\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.772913 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.772627    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-tls\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.772913 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.772655    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04b2f1f3-f80c-4137-973d-e789d2c488ce-sys\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.772913 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.772713    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-textfile\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.773075 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.773023    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04b2f1f3-f80c-4137-973d-e789d2c488ce-root\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.773127 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.773071    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-textfile\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.773127 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.773086    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-wtmp\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.773231 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.773162    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04b2f1f3-f80c-4137-973d-e789d2c488ce-sys\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.773489 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:46:28.773288    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 30 03:46:28.773489 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:46:28.773346    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-tls podName:04b2f1f3-f80c-4137-973d-e789d2c488ce nodeName:}" failed. No retries permitted until 2026-07-30 03:46:29.273328322 +0000 UTC m=+88.312039338 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-tls") pod "node-exporter-qd6jz" (UID: "04b2f1f3-f80c-4137-973d-e789d2c488ce") : secret "node-exporter-tls" not found
Jul 30 03:46:28.773700 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.773493    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04b2f1f3-f80c-4137-973d-e789d2c488ce-metrics-client-ca\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.773700 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.773523    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-accelerators-collector-config\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.775964 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.775941    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.783586 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.783443    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f7vk\" (UniqueName: \"kubernetes.io/projected/04b2f1f3-f80c-4137-973d-e789d2c488ce-kube-api-access-6f7vk\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:28.808381 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.808354    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:28.908366 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.908327    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:28.926421 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.926372    2569 patch_prober.go:28] interesting pod/image-registry-6f9c8fcc6c-hpvwz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 30 03:46:28.926600 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:28.926430    2569 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz" podUID="9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 03:46:29.008865 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:29.008836    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:29.112586 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:29.112457    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:29.233244 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:29.233195    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:29.277802 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:29.277693    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-tls\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:29.280392 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:29.280370    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04b2f1f3-f80c-4137-973d-e789d2c488ce-node-exporter-tls\") pod \"node-exporter-qd6jz\" (UID: \"04b2f1f3-f80c-4137-973d-e789d2c488ce\") " pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:29.337242 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:29.337172    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:29.489496 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:29.489415    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qd6jz"
Jul 30 03:46:29.524940 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:29.524902    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:29.611894 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:29.611847    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:30.117101 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:30.117065    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:30.512423 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:46:30.512382    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b2f1f3_f80c_4137_973d_e789d2c488ce.slice/crio-3012372494ced118ed239393eda9af55ae086b64071195d4bfbf4d2618d1224f WatchSource:0}: Error finding container 3012372494ced118ed239393eda9af55ae086b64071195d4bfbf4d2618d1224f: Status 404 returned error can't find the container with id 3012372494ced118ed239393eda9af55ae086b64071195d4bfbf4d2618d1224f
Jul 30 03:46:30.803431 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:30.803405    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:46:30.903126 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:30.903092    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qd6jz" event={"ID":"04b2f1f3-f80c-4137-973d-e789d2c488ce","Type":"ContainerStarted","Data":"3012372494ced118ed239393eda9af55ae086b64071195d4bfbf4d2618d1224f"}
Jul 30 03:46:30.904905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:30.904883    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-88l4r" event={"ID":"ab25f6a4-deba-4c63-a0a7-3be8b62d67c2","Type":"ContainerStarted","Data":"eb4cf3d219467266f4772af19d870b3147bb8640d3d2fccfaf4b579810decf01"}
Jul 30 03:46:30.920742 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:30.920697    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-88l4r" podStartSLOduration=33.252060366 podStartE2EDuration="44.92067813s" totalImagesPullingTime="11.668617764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:45:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:30.919393483 +0000 UTC m=+89.958104502" watchObservedRunningTime="2026-07-30 03:46:30.92067813 +0000 UTC m=+89.959389149"
Jul 30 03:46:31.327113 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:31.327048    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:31.373159 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:31.373135    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:31.501886 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:31.501834    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:31.556676 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:31.556649    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:31.687577 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:31.687505    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:31.823027 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:31.822999    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:31.909712 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:31.909679    2569 generic.go:356] "Generic (PLEG): container finished" podID="04b2f1f3-f80c-4137-973d-e789d2c488ce" containerID="c39b0d011b8c83cfbc1792d9804d164a7ade21b4d92e93abbf2ea57082e88940" exitCode=0
Jul 30 03:46:31.909844 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:31.909767    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qd6jz" event={"ID":"04b2f1f3-f80c-4137-973d-e789d2c488ce","Type":"ContainerDied","Data":"c39b0d011b8c83cfbc1792d9804d164a7ade21b4d92e93abbf2ea57082e88940"}
Jul 30 03:46:31.923589 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:31.923567    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:32.196263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:32.196182    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"]
Jul 30 03:46:32.267044 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:32.267014    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:32.322010 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:32.321985    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:32.915613 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:32.915580    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qd6jz" event={"ID":"04b2f1f3-f80c-4137-973d-e789d2c488ce","Type":"ContainerStarted","Data":"33a40cf19630c0fa308692a170015aa04082a9f753fe01e5455038e7dc39b3f1"}
Jul 30 03:46:32.915977 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:32.915633    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qd6jz" event={"ID":"04b2f1f3-f80c-4137-973d-e789d2c488ce","Type":"ContainerStarted","Data":"7007fd506b6ea61a8878e6ad8f67de96b255fdc4a6ce68bcde3087eb2fe9557d"}
Jul 30 03:46:32.935082 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:32.935037    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qd6jz" podStartSLOduration=3.967316105 podStartE2EDuration="4.93502094s" totalImagesPullingTime="897.084716ms" totalInitContainerRuntime="70.620119ms" isStatefulPod=true podCreationTimestamp="2026-07-30 03:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:32.93304122 +0000 UTC m=+91.971752238" watchObservedRunningTime="2026-07-30 03:46:32.93502094 +0000 UTC m=+91.973731958"
Jul 30 03:46:32.937795 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:32.937759    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:32.993122 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:32.993054    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:33.324126 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:33.324086    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:33.499855 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:33.499817    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:34.769997 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:34.769963    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:35.460012 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:35.459981    2569 scope.go:117] "RemoveContainer" containerID="1914330f900c5a5b49ff6a4d7687fb24bfb3fa9b9bbde4beff9eb1d53f477fef"
Jul 30 03:46:35.930883 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:35.930853    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 03:46:35.931230 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:35.930964    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v" event={"ID":"30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0","Type":"ContainerStarted","Data":"43830b4e2ba38bb6af3f868b6387e5437f5c85c2b1edf5d4ae62c48af3877542"}
Jul 30 03:46:35.931296 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:35.931276    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:46:35.945953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:35.945882    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v" podStartSLOduration=51.822254878 podStartE2EDuration="54.945865482s" totalImagesPullingTime="3.123610604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:35.945628558 +0000 UTC m=+94.984339578" watchObservedRunningTime="2026-07-30 03:46:35.945865482 +0000 UTC m=+94.984576498"
Jul 30 03:46:36.094116 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.094083    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-tth9v"
Jul 30 03:46:36.269005 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.268974    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-zsxjn"]
Jul 30 03:46:36.272221 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.272202    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-zsxjn"
Jul 30 03:46:36.274276 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.274258    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 30 03:46:36.274353 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.274303    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 30 03:46:36.274353 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.274319    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-l6bt8\""
Jul 30 03:46:36.280776 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.280756    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-zsxjn"]
Jul 30 03:46:36.289580 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.289554    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:36.384149 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.384123    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:36.440103 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.440077    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flsz2\" (UniqueName: \"kubernetes.io/projected/a2ffbdcb-2be1-4bf6-8ab7-1539111e21cc-kube-api-access-flsz2\") pod \"downloads-7c97dc4555-zsxjn\" (UID: \"a2ffbdcb-2be1-4bf6-8ab7-1539111e21cc\") " pod="openshift-console/downloads-7c97dc4555-zsxjn"
Jul 30 03:46:36.485455 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.485427    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:36.540544 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.540476    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flsz2\" (UniqueName: \"kubernetes.io/projected/a2ffbdcb-2be1-4bf6-8ab7-1539111e21cc-kube-api-access-flsz2\") pod \"downloads-7c97dc4555-zsxjn\" (UID: \"a2ffbdcb-2be1-4bf6-8ab7-1539111e21cc\") " pod="openshift-console/downloads-7c97dc4555-zsxjn"
Jul 30 03:46:36.547653 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.547636    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flsz2\" (UniqueName: \"kubernetes.io/projected/a2ffbdcb-2be1-4bf6-8ab7-1539111e21cc-kube-api-access-flsz2\") pod \"downloads-7c97dc4555-zsxjn\" (UID: \"a2ffbdcb-2be1-4bf6-8ab7-1539111e21cc\") " pod="openshift-console/downloads-7c97dc4555-zsxjn"
Jul 30 03:46:36.582596 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.582575    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-zsxjn"
Jul 30 03:46:36.709747 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.709719    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-zsxjn"]
Jul 30 03:46:36.712785 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:46:36.712749    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ffbdcb_2be1_4bf6_8ab7_1539111e21cc.slice/crio-06a89eba3547994b81070764998687edb95ce5f050e36543f9acfde2ffaf8192 WatchSource:0}: Error finding container 06a89eba3547994b81070764998687edb95ce5f050e36543f9acfde2ffaf8192: Status 404 returned error can't find the container with id 06a89eba3547994b81070764998687edb95ce5f050e36543f9acfde2ffaf8192
Jul 30 03:46:36.758234 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.758180    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:36.858791 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.858733    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:36.935354 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.935328    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-zsxjn" event={"ID":"a2ffbdcb-2be1-4bf6-8ab7-1539111e21cc","Type":"ContainerStarted","Data":"06a89eba3547994b81070764998687edb95ce5f050e36543f9acfde2ffaf8192"}
Jul 30 03:46:36.958588 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:36.958559    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:37.009919 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:37.009893    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:37.152014 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:37.151945    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:37.253074 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:37.253041    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:38.172654 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:38.172610    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:38.236130 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:38.236093    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:38.337082 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:38.337048    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:38.388586 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:38.388555    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:41.944077 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:41.944036    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:42.978550 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:42.978495    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:43.031310 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:43.031274    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:43.823710 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:43.823679    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bpwlj"
Jul 30 03:46:46.117797 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:46.117752    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:46.218194 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:46.218163    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:46.679007 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:46.678974    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:46.776815 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:46.776783    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:47.514491 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.514452    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67bc4fbc9d-pv9ng"]
Jul 30 03:46:47.551315 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.551283    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67bc4fbc9d-pv9ng"]
Jul 30 03:46:47.551466 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.551411    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.553846 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.553824    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ncfl2\""
Jul 30 03:46:47.553985 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.553824    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 30 03:46:47.554593 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.554423    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 30 03:46:47.554593 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.554442    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 30 03:46:47.554593 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.554476    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 30 03:46:47.554593 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.554508    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 30 03:46:47.629320 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.629286    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-oauth-serving-cert\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.629459 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.629340    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-service-ca\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.629459 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.629381    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8szt9\" (UniqueName: \"kubernetes.io/projected/4fcf6c11-acd7-4db0-853d-0157b1e35da3-kube-api-access-8szt9\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.629459 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.629413    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-serving-cert\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.629459 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.629432    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-oauth-config\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.629650 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.629489    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-config\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.730866 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.730833    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8szt9\" (UniqueName: \"kubernetes.io/projected/4fcf6c11-acd7-4db0-853d-0157b1e35da3-kube-api-access-8szt9\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.731018 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.730893    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-serving-cert\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.731018 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.730914    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-oauth-config\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.731018 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.730945    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-config\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.731018 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.730993    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-oauth-serving-cert\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.731246 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.731036    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-service-ca\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.731786 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.731738    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-config\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.731786 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.731767    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-oauth-serving-cert\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.732088 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.731880    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-service-ca\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.733786 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.733760    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-oauth-config\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.734000 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.733982    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-serving-cert\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.739634 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.739609    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8szt9\" (UniqueName: \"kubernetes.io/projected/4fcf6c11-acd7-4db0-853d-0157b1e35da3-kube-api-access-8szt9\") pod \"console-67bc4fbc9d-pv9ng\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") " pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.838923 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.838833    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:47.863652 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.863622    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:46:47.935667 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:47.935636    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:49.594987 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.594954    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84cc994789-mbnkd"]
Jul 30 03:46:49.615951 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.615926    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84cc994789-mbnkd"]
Jul 30 03:46:49.616111 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.616024    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.622612 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.621958    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:49.639396 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.639373    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 30 03:46:49.647890 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.647857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-service-ca\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.647991 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.647902    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-oauth-serving-cert\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.647991 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.647933    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-console-config\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.647991 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.647960    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng8ts\" (UniqueName: \"kubernetes.io/projected/584e5576-0b0f-450b-b079-eacd575d16c4-kube-api-access-ng8ts\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.648130 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.647989    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-trusted-ca-bundle\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.648130 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.648036    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/584e5576-0b0f-450b-b079-eacd575d16c4-console-serving-cert\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.648130 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.648060    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/584e5576-0b0f-450b-b079-eacd575d16c4-console-oauth-config\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.720828 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.720761    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:49.748583 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.748560    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/584e5576-0b0f-450b-b079-eacd575d16c4-console-serving-cert\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.748583 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.748589    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/584e5576-0b0f-450b-b079-eacd575d16c4-console-oauth-config\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.748784 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.748629    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-service-ca\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.748784 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.748653    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-oauth-serving-cert\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.748784 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.748673    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-console-config\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.748784 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.748691    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng8ts\" (UniqueName: \"kubernetes.io/projected/584e5576-0b0f-450b-b079-eacd575d16c4-kube-api-access-ng8ts\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.748784 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.748724    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-trusted-ca-bundle\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.749589 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.749558    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-oauth-serving-cert\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.749706 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.749639    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-service-ca\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.749825 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.749746    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-trusted-ca-bundle\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.749886 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.749827    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-console-config\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.751649 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.751624    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/584e5576-0b0f-450b-b079-eacd575d16c4-console-serving-cert\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.751759 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.751649    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/584e5576-0b0f-450b-b079-eacd575d16c4-console-oauth-config\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.757360 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.757332    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng8ts\" (UniqueName: \"kubernetes.io/projected/584e5576-0b0f-450b-b079-eacd575d16c4-kube-api-access-ng8ts\") pod \"console-84cc994789-mbnkd\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") " pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:49.930873 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:49.930782    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:46:50.192861 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:50.192763    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:50.292384 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:50.292346    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:52.062643 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:52.062607    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:52.161507 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:52.161470    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:54.086208 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:54.086136    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67bc4fbc9d-pv9ng"]
Jul 30 03:46:54.104401 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:54.104377    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84cc994789-mbnkd"]
Jul 30 03:46:54.230448 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:46:54.230372    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fcf6c11_acd7_4db0_853d_0157b1e35da3.slice/crio-a130ef830a8bddcf8cca29e4ec7cb1364a7a28d4729470c4b0c8195e3b6be988 WatchSource:0}: Error finding container a130ef830a8bddcf8cca29e4ec7cb1364a7a28d4729470c4b0c8195e3b6be988: Status 404 returned error can't find the container with id a130ef830a8bddcf8cca29e4ec7cb1364a7a28d4729470c4b0c8195e3b6be988
Jul 30 03:46:54.231380 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:46:54.231345    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod584e5576_0b0f_450b_b079_eacd575d16c4.slice/crio-e42315a556d104eb6854cd5cb84fc1a23c6ec0ec89b4fdb65d0a710d5e436a50 WatchSource:0}: Error finding container e42315a556d104eb6854cd5cb84fc1a23c6ec0ec89b4fdb65d0a710d5e436a50: Status 404 returned error can't find the container with id e42315a556d104eb6854cd5cb84fc1a23c6ec0ec89b4fdb65d0a710d5e436a50
Jul 30 03:46:54.389640 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:54.389599    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:54.488468 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:54.488402    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:55.006619 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:55.006580    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84cc994789-mbnkd" event={"ID":"584e5576-0b0f-450b-b079-eacd575d16c4","Type":"ContainerStarted","Data":"e42315a556d104eb6854cd5cb84fc1a23c6ec0ec89b4fdb65d0a710d5e436a50"}
Jul 30 03:46:55.009328 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:55.009294    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bc4fbc9d-pv9ng" event={"ID":"4fcf6c11-acd7-4db0-853d-0157b1e35da3","Type":"ContainerStarted","Data":"a130ef830a8bddcf8cca29e4ec7cb1364a7a28d4729470c4b0c8195e3b6be988"}
Jul 30 03:46:55.011608 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:55.011565    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-zsxjn" event={"ID":"a2ffbdcb-2be1-4bf6-8ab7-1539111e21cc","Type":"ContainerStarted","Data":"d83367854d147b43a73c77722a1d6ab2805ae0cd8b24ddedea6053e9294065a6"}
Jul 30 03:46:55.011955 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:55.011931    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-zsxjn"
Jul 30 03:46:55.024847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:55.024826    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-zsxjn"
Jul 30 03:46:55.052746 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:55.052693    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-zsxjn" podStartSLOduration=1.467754686 podStartE2EDuration="19.052673405s" totalImagesPullingTime="17.584918719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:55.033710658 +0000 UTC m=+114.072421676" watchObservedRunningTime="2026-07-30 03:46:55.052673405 +0000 UTC m=+114.091384425"
Jul 30 03:46:57.215468 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:57.215426    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz" podUID="9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" containerName="registry" containerID="cri-o://3fe6585c853bd918876cc4a98739203f67f31c0787f50d7def7f58bbb16d9f77" gracePeriod=30
Jul 30 03:46:58.611758 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.611731    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:46:58.724308 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.724229    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-ca-trust-extracted\") pod \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") "
Jul 30 03:46:58.724308 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.724285    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-bound-sa-token\") pod \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") "
Jul 30 03:46:58.724550 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.724327    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqh82\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-kube-api-access-fqh82\") pod \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") "
Jul 30 03:46:58.724550 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.724368    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-installation-pull-secrets\") pod \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") "
Jul 30 03:46:58.724842 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.724815    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls\") pod \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") "
Jul 30 03:46:58.724926 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.724870    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-trusted-ca\") pod \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") "
Jul 30 03:46:58.724926 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.724898    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-certificates\") pod \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") "
Jul 30 03:46:58.725041 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.724942    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-image-registry-private-configuration\") pod \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\" (UID: \"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca\") "
Jul 30 03:46:58.725834 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.725679    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" (UID: "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:46:58.726570 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.726100    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" (UID: "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:46:58.727368 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.727339    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" (UID: "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:46:58.730845 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.730815    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-kube-api-access-fqh82" (OuterVolumeSpecName: "kube-api-access-fqh82") pod "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" (UID: "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca"). InnerVolumeSpecName "kube-api-access-fqh82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:46:58.731070 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.730903    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" (UID: "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:46:58.731245 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.731215    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" (UID: "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:46:58.731430 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.731240    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" (UID: "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:46:58.737614 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.737586    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" (UID: "9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:46:58.826857 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.826821    2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-installation-pull-secrets\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:46:58.826857 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.826851    2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:46:58.826857 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.826861    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-trusted-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:46:58.826857 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.826869    2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-registry-certificates\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:46:58.827139 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.826878    2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-image-registry-private-configuration\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:46:58.827139 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.826887    2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-ca-trust-extracted\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:46:58.827139 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.826900    2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-bound-sa-token\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:46:58.827139 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:58.826915    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqh82\" (UniqueName: \"kubernetes.io/projected/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca-kube-api-access-fqh82\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:46:59.034750 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:59.033273    2569 generic.go:356] "Generic (PLEG): container finished" podID="9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" containerID="3fe6585c853bd918876cc4a98739203f67f31c0787f50d7def7f58bbb16d9f77" exitCode=0
Jul 30 03:46:59.034750 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:59.033359    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz" event={"ID":"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca","Type":"ContainerDied","Data":"3fe6585c853bd918876cc4a98739203f67f31c0787f50d7def7f58bbb16d9f77"}
Jul 30 03:46:59.034750 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:59.033395    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz" event={"ID":"9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca","Type":"ContainerDied","Data":"e166be2460b1cea12764e7b8b7496d529d24c41b4c99776fbdf532c6398cd385"}
Jul 30 03:46:59.034750 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:59.033417    2569 scope.go:117] "RemoveContainer" containerID="3fe6585c853bd918876cc4a98739203f67f31c0787f50d7def7f58bbb16d9f77"
Jul 30 03:46:59.034750 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:59.033587    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"
Jul 30 03:46:59.047682 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:59.047655    2569 scope.go:117] "RemoveContainer" containerID="3fe6585c853bd918876cc4a98739203f67f31c0787f50d7def7f58bbb16d9f77"
Jul 30 03:46:59.047964 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:46:59.047934    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fe6585c853bd918876cc4a98739203f67f31c0787f50d7def7f58bbb16d9f77\": container with ID starting with 3fe6585c853bd918876cc4a98739203f67f31c0787f50d7def7f58bbb16d9f77 not found: ID does not exist" containerID="3fe6585c853bd918876cc4a98739203f67f31c0787f50d7def7f58bbb16d9f77"
Jul 30 03:46:59.048052 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:59.047978    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fe6585c853bd918876cc4a98739203f67f31c0787f50d7def7f58bbb16d9f77"} err="failed to get container status \"3fe6585c853bd918876cc4a98739203f67f31c0787f50d7def7f58bbb16d9f77\": rpc error: code = NotFound desc = could not find container \"3fe6585c853bd918876cc4a98739203f67f31c0787f50d7def7f58bbb16d9f77\": container with ID starting with 3fe6585c853bd918876cc4a98739203f67f31c0787f50d7def7f58bbb16d9f77 not found: ID does not exist"
Jul 30 03:46:59.069633 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:59.069612    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"]
Jul 30 03:46:59.070759 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:59.070576    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6f9c8fcc6c-hpvwz"]
Jul 30 03:46:59.465606 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:46:59.465573    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" path="/var/lib/kubelet/pods/9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca/volumes"
Jul 30 03:47:00.040523 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:00.040482    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84cc994789-mbnkd" event={"ID":"584e5576-0b0f-450b-b079-eacd575d16c4","Type":"ContainerStarted","Data":"afff1af79f8caef39cd105b8e27fa746f301786dd08b63bab3da790856be341b"}
Jul 30 03:47:00.042574 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:00.042524    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bc4fbc9d-pv9ng" event={"ID":"4fcf6c11-acd7-4db0-853d-0157b1e35da3","Type":"ContainerStarted","Data":"a254668b2ff3c3fd0545ad4f7baba1d51c6b642f70f824d9578afd3863266d2b"}
Jul 30 03:47:00.057739 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:00.057684    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84cc994789-mbnkd" podStartSLOduration=6.331875554 podStartE2EDuration="11.057663011s" totalImagesPullingTime="4.725787457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:46:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:47:00.057031472 +0000 UTC m=+119.095742496" watchObservedRunningTime="2026-07-30 03:47:00.057663011 +0000 UTC m=+119.096374041"
Jul 30 03:47:00.072816 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:00.072768    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67bc4fbc9d-pv9ng" podStartSLOduration=8.491327661 podStartE2EDuration="13.072725396s" totalImagesPullingTime="4.581397735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:46:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:47:00.071847033 +0000 UTC m=+119.110558052" watchObservedRunningTime="2026-07-30 03:47:00.072725396 +0000 UTC m=+119.111436428"
Jul 30 03:47:01.671363 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:01.671279    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:01.722094 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:01.722055    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:04.587023 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:04.586988    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:04.642569 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:04.642519    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:06.066203 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:06.066128    2569 generic.go:356] "Generic (PLEG): container finished" podID="e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2" containerID="6a3022c9d1c60d0deab9923381f80256eb1577b159460572b316dec260aebc56" exitCode=0
Jul 30 03:47:06.066203 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:06.066178    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-v7mn4" event={"ID":"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2","Type":"ContainerDied","Data":"6a3022c9d1c60d0deab9923381f80256eb1577b159460572b316dec260aebc56"}
Jul 30 03:47:06.066702 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:06.066451    2569 scope.go:117] "RemoveContainer" containerID="6a3022c9d1c60d0deab9923381f80256eb1577b159460572b316dec260aebc56"
Jul 30 03:47:07.071978 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:07.071941    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-v7mn4" event={"ID":"e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2","Type":"ContainerStarted","Data":"6e6f26de79254fe2156e41fe8364ccc8f16c2b0ae656e7fd52b6cabefab6e138"}
Jul 30 03:47:07.863758 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:07.863719    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:47:07.863977 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:07.863780    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:47:07.868360 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:07.868337    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:47:08.078967 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:08.078942    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:47:08.195486 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:08.195400    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:08.244471 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:08.244437    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:08.780731 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:08.780689    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:08.882598 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:08.882561    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:08.934773 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:08.934741    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:08.985671 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:08.985642    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:09.115585 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:09.115497    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ltv96_e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7/cluster-monitoring-operator/0.log"
Jul 30 03:47:09.931176 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:09.931137    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:47:09.931362 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:09.931244    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:47:09.935784 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:09.935760    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:47:10.085161 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:10.085135    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:47:10.128036 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:10.128007    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67bc4fbc9d-pv9ng"]
Jul 30 03:47:10.263922 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:10.263845    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:10.369657 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:10.369622    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:10.469775 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:10.469745    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:11.514207 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:11.514178    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qd6jz_04b2f1f3-f80c-4137-973d-e789d2c488ce/init-textfile/0.log"
Jul 30 03:47:11.714953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:11.714925    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qd6jz_04b2f1f3-f80c-4137-973d-e789d2c488ce/node-exporter/0.log"
Jul 30 03:47:11.914203 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:11.914174    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qd6jz_04b2f1f3-f80c-4137-973d-e789d2c488ce/kube-rbac-proxy/0.log"
Jul 30 03:47:16.513997 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:16.513966    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-tcpmc_56a7549c-0cff-4485-8518-30d431954079/networking-console-plugin/0.log"
Jul 30 03:47:16.664789 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:16.664761    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:16.714789 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:16.714766    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 03:47:16.719137 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:16.719113    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:16.916524 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:16.916495    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/3.log"
Jul 30 03:47:17.117147 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:17.117116    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-67bc4fbc9d-pv9ng_4fcf6c11-acd7-4db0-853d-0157b1e35da3/console/0.log"
Jul 30 03:47:17.314584 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:17.314563    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-84cc994789-mbnkd_584e5576-0b0f-450b-b079-eacd575d16c4/console/0.log"
Jul 30 03:47:17.516515 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:17.516487    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-zsxjn_a2ffbdcb-2be1-4bf6-8ab7-1539111e21cc/download-server/0.log"
Jul 30 03:47:18.314177 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:18.314151    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lzf7c_ad3a8ea4-d784-43ae-832c-16a6ebf99c1f/serve-healthcheck-canary/0.log"
Jul 30 03:47:35.147962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.147889    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67bc4fbc9d-pv9ng" podUID="4fcf6c11-acd7-4db0-853d-0157b1e35da3" containerName="console" containerID="cri-o://a254668b2ff3c3fd0545ad4f7baba1d51c6b642f70f824d9578afd3863266d2b" gracePeriod=15
Jul 30 03:47:35.414358 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.414306    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:47:35.509197 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.509167    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-service-ca\") pod \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") "
Jul 30 03:47:35.509361 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.509214    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-oauth-config\") pod \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") "
Jul 30 03:47:35.509361 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.509256    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-serving-cert\") pod \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") "
Jul 30 03:47:35.509361 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.509272    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-config\") pod \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") "
Jul 30 03:47:35.509361 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.509288    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-oauth-serving-cert\") pod \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") "
Jul 30 03:47:35.509361 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.509304    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8szt9\" (UniqueName: \"kubernetes.io/projected/4fcf6c11-acd7-4db0-853d-0157b1e35da3-kube-api-access-8szt9\") pod \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\" (UID: \"4fcf6c11-acd7-4db0-853d-0157b1e35da3\") "
Jul 30 03:47:35.509668 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.509642    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-service-ca" (OuterVolumeSpecName: "service-ca") pod "4fcf6c11-acd7-4db0-853d-0157b1e35da3" (UID: "4fcf6c11-acd7-4db0-853d-0157b1e35da3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:47:35.509761 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.509734    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-config" (OuterVolumeSpecName: "console-config") pod "4fcf6c11-acd7-4db0-853d-0157b1e35da3" (UID: "4fcf6c11-acd7-4db0-853d-0157b1e35da3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:47:35.509819 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.509742    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4fcf6c11-acd7-4db0-853d-0157b1e35da3" (UID: "4fcf6c11-acd7-4db0-853d-0157b1e35da3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:47:35.511753 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.511729    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4fcf6c11-acd7-4db0-853d-0157b1e35da3" (UID: "4fcf6c11-acd7-4db0-853d-0157b1e35da3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:47:35.511839 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.511741    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4fcf6c11-acd7-4db0-853d-0157b1e35da3" (UID: "4fcf6c11-acd7-4db0-853d-0157b1e35da3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:47:35.511839 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.511767    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fcf6c11-acd7-4db0-853d-0157b1e35da3-kube-api-access-8szt9" (OuterVolumeSpecName: "kube-api-access-8szt9") pod "4fcf6c11-acd7-4db0-853d-0157b1e35da3" (UID: "4fcf6c11-acd7-4db0-853d-0157b1e35da3"). InnerVolumeSpecName "kube-api-access-8szt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:47:35.609751 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.609724    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-oauth-config\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:47:35.609751 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.609747    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-serving-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:47:35.609894 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.609758    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-console-config\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:47:35.609894 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.609767    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-oauth-serving-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:47:35.609894 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.609776    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8szt9\" (UniqueName: \"kubernetes.io/projected/4fcf6c11-acd7-4db0-853d-0157b1e35da3-kube-api-access-8szt9\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:47:35.609894 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:35.609784    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fcf6c11-acd7-4db0-853d-0157b1e35da3-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:47:36.177760 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:36.177728    2569 generic.go:356] "Generic (PLEG): container finished" podID="4fcf6c11-acd7-4db0-853d-0157b1e35da3" containerID="a254668b2ff3c3fd0545ad4f7baba1d51c6b642f70f824d9578afd3863266d2b" exitCode=0
Jul 30 03:47:36.178164 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:36.177777    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bc4fbc9d-pv9ng"
Jul 30 03:47:36.178164 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:36.177794    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bc4fbc9d-pv9ng" event={"ID":"4fcf6c11-acd7-4db0-853d-0157b1e35da3","Type":"ContainerDied","Data":"a254668b2ff3c3fd0545ad4f7baba1d51c6b642f70f824d9578afd3863266d2b"}
Jul 30 03:47:36.178164 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:36.177836    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bc4fbc9d-pv9ng" event={"ID":"4fcf6c11-acd7-4db0-853d-0157b1e35da3","Type":"ContainerDied","Data":"a130ef830a8bddcf8cca29e4ec7cb1364a7a28d4729470c4b0c8195e3b6be988"}
Jul 30 03:47:36.178164 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:36.177855    2569 scope.go:117] "RemoveContainer" containerID="a254668b2ff3c3fd0545ad4f7baba1d51c6b642f70f824d9578afd3863266d2b"
Jul 30 03:47:36.186356 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:36.186337    2569 scope.go:117] "RemoveContainer" containerID="a254668b2ff3c3fd0545ad4f7baba1d51c6b642f70f824d9578afd3863266d2b"
Jul 30 03:47:36.186642 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:47:36.186622    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a254668b2ff3c3fd0545ad4f7baba1d51c6b642f70f824d9578afd3863266d2b\": container with ID starting with a254668b2ff3c3fd0545ad4f7baba1d51c6b642f70f824d9578afd3863266d2b not found: ID does not exist" containerID="a254668b2ff3c3fd0545ad4f7baba1d51c6b642f70f824d9578afd3863266d2b"
Jul 30 03:47:36.186700 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:36.186650    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a254668b2ff3c3fd0545ad4f7baba1d51c6b642f70f824d9578afd3863266d2b"} err="failed to get container status \"a254668b2ff3c3fd0545ad4f7baba1d51c6b642f70f824d9578afd3863266d2b\": rpc error: code = NotFound desc = could not find container \"a254668b2ff3c3fd0545ad4f7baba1d51c6b642f70f824d9578afd3863266d2b\": container with ID starting with a254668b2ff3c3fd0545ad4f7baba1d51c6b642f70f824d9578afd3863266d2b not found: ID does not exist"
Jul 30 03:47:36.196469 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:36.196440    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67bc4fbc9d-pv9ng"]
Jul 30 03:47:36.200292 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:36.200271    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67bc4fbc9d-pv9ng"]
Jul 30 03:47:37.464574 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:37.464520    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fcf6c11-acd7-4db0-853d-0157b1e35da3" path="/var/lib/kubelet/pods/4fcf6c11-acd7-4db0-853d-0157b1e35da3/volumes"
Jul 30 03:47:44.628855 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:44.628818    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:44.681000 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:44.680970    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:56.530465 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:56.530352    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:56.581464 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:47:56.581427    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:01.679130 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:01.679032    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:01.780222 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:01.780181    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:01.829038 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:01.828999    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:13.799240 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.799202    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5db69bb768-wc6c8"]
Jul 30 03:48:13.799747 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.799495    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" containerName="registry"
Jul 30 03:48:13.799747 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.799508    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" containerName="registry"
Jul 30 03:48:13.799747 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.799515    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fcf6c11-acd7-4db0-853d-0157b1e35da3" containerName="console"
Jul 30 03:48:13.799747 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.799521    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fcf6c11-acd7-4db0-853d-0157b1e35da3" containerName="console"
Jul 30 03:48:13.799747 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.799598    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b0bf04d-3aa6-416c-9b06-cc82ea8fb2ca" containerName="registry"
Jul 30 03:48:13.799747 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.799607    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fcf6c11-acd7-4db0-853d-0157b1e35da3" containerName="console"
Jul 30 03:48:13.801985 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.801962    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.813471 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.813442    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db69bb768-wc6c8"]
Jul 30 03:48:13.854427 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.854397    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:13.871434 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.871410    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-oauth-serving-cert\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.871575 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.871440    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-service-ca\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.871575 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.871474    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znf9c\" (UniqueName: \"kubernetes.io/projected/fcbc016d-220a-40ad-b991-6f446d37fdb9-kube-api-access-znf9c\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.871575 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.871525    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-serving-cert\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.871575 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.871571    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-oauth-config\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.871729 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.871598    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-trusted-ca-bundle\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.871729 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.871625    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-config\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.972470 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.972433    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-oauth-serving-cert\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.972470 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.972476    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-service-ca\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.972679 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.972510    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znf9c\" (UniqueName: \"kubernetes.io/projected/fcbc016d-220a-40ad-b991-6f446d37fdb9-kube-api-access-znf9c\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.972679 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.972556    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-serving-cert\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.972679 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.972574    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-oauth-config\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.972679 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.972600    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-trusted-ca-bundle\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.972679 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.972634    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-config\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.973314 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.973287    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-service-ca\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.973422 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.973287    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-oauth-serving-cert\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.973422 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.973372    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-trusted-ca-bundle\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.973499 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.973358    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-config\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.975143 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.975115    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-serving-cert\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.975233 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.975144    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-oauth-config\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:13.981104 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:13.981077    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znf9c\" (UniqueName: \"kubernetes.io/projected/fcbc016d-220a-40ad-b991-6f446d37fdb9-kube-api-access-znf9c\") pod \"console-5db69bb768-wc6c8\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") " pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:14.114321 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:14.114240    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:14.240319 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:14.240286    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db69bb768-wc6c8"]
Jul 30 03:48:14.243478 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:48:14.243448    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcbc016d_220a_40ad_b991_6f446d37fdb9.slice/crio-dc94118bfa5f3e60934184ffe367f5957ce8d0b9e4477a76b8eb7c442979be3f WatchSource:0}: Error finding container dc94118bfa5f3e60934184ffe367f5957ce8d0b9e4477a76b8eb7c442979be3f: Status 404 returned error can't find the container with id dc94118bfa5f3e60934184ffe367f5957ce8d0b9e4477a76b8eb7c442979be3f
Jul 30 03:48:14.315453 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:14.315419    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db69bb768-wc6c8" event={"ID":"fcbc016d-220a-40ad-b991-6f446d37fdb9","Type":"ContainerStarted","Data":"19c471238f0e3567777634972945817fde8ab115e4cdbf015025d10a4b45375e"}
Jul 30 03:48:14.315597 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:14.315468    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db69bb768-wc6c8" event={"ID":"fcbc016d-220a-40ad-b991-6f446d37fdb9","Type":"ContainerStarted","Data":"dc94118bfa5f3e60934184ffe367f5957ce8d0b9e4477a76b8eb7c442979be3f"}
Jul 30 03:48:14.331004 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:14.330952    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5db69bb768-wc6c8" podStartSLOduration=1.330935489 podStartE2EDuration="1.330935489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:48:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:48:14.330489504 +0000 UTC m=+193.369200516" watchObservedRunningTime="2026-07-30 03:48:14.330935489 +0000 UTC m=+193.369646509"
Jul 30 03:48:14.461837 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:14.461742    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:17.920292 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:17.920263    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:18.020240 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.020208    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:18.517743 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.517716    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"]
Jul 30 03:48:18.519870 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.519854    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"
Jul 30 03:48:18.521734 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.521716    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 30 03:48:18.521840 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.521813    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dqq9q\""
Jul 30 03:48:18.522313 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.522299    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 30 03:48:18.529279 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.529256    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"]
Jul 30 03:48:18.608175 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.608142    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6\" (UID: \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"
Jul 30 03:48:18.608329 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.608209    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkwp8\" (UniqueName: \"kubernetes.io/projected/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-kube-api-access-bkwp8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6\" (UID: \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"
Jul 30 03:48:18.608329 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.608232    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6\" (UID: \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"
Jul 30 03:48:18.709480 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.709449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkwp8\" (UniqueName: \"kubernetes.io/projected/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-kube-api-access-bkwp8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6\" (UID: \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"
Jul 30 03:48:18.709480 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.709484    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6\" (UID: \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"
Jul 30 03:48:18.709670 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.709510    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6\" (UID: \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"
Jul 30 03:48:18.709838 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.709820    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6\" (UID: \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"
Jul 30 03:48:18.709873 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.709848    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6\" (UID: \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"
Jul 30 03:48:18.716879 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.716854    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkwp8\" (UniqueName: \"kubernetes.io/projected/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-kube-api-access-bkwp8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6\" (UID: \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"
Jul 30 03:48:18.830193 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.830136    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"
Jul 30 03:48:18.953128 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:18.953095    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"]
Jul 30 03:48:18.955925 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:48:18.955903    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12dea14e_b49c_4a04_9bd9_57bb4b4f5a89.slice/crio-277189ad85e0df8c479dfa8f0594cd0b4e89cd906cd17259b385ef42f8e4752f WatchSource:0}: Error finding container 277189ad85e0df8c479dfa8f0594cd0b4e89cd906cd17259b385ef42f8e4752f: Status 404 returned error can't find the container with id 277189ad85e0df8c479dfa8f0594cd0b4e89cd906cd17259b385ef42f8e4752f
Jul 30 03:48:19.334715 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:19.334677    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6" event={"ID":"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89","Type":"ContainerStarted","Data":"277189ad85e0df8c479dfa8f0594cd0b4e89cd906cd17259b385ef42f8e4752f"}
Jul 30 03:48:24.114746 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:24.114693    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:24.115171 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:24.114803    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:24.121151 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:24.121123    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:24.359074 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:24.359039    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:48:24.401081 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:24.400998    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84cc994789-mbnkd"]
Jul 30 03:48:24.527697 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:24.527659    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:24.622165 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:24.622132    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:24.673688 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:24.673607    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:25.359978 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:25.359948    2569 generic.go:356] "Generic (PLEG): container finished" podID="12dea14e-b49c-4a04-9bd9-57bb4b4f5a89" containerID="8a8586baebd32c44e72a0f32f4df5f8ad5725b79261f1c3dd7acbd0725dfb66f" exitCode=0
Jul 30 03:48:25.360356 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:25.360045    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6" event={"ID":"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89","Type":"ContainerDied","Data":"8a8586baebd32c44e72a0f32f4df5f8ad5725b79261f1c3dd7acbd0725dfb66f"}
Jul 30 03:48:28.373752 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:28.373719    2569 generic.go:356] "Generic (PLEG): container finished" podID="12dea14e-b49c-4a04-9bd9-57bb4b4f5a89" containerID="b83d2231f8cf980d99732b465e089868ea5520291b45a22ba1e723ed5457e93c" exitCode=0
Jul 30 03:48:28.374114 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:28.373767    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6" event={"ID":"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89","Type":"ContainerDied","Data":"b83d2231f8cf980d99732b465e089868ea5520291b45a22ba1e723ed5457e93c"}
Jul 30 03:48:36.404953 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:36.404916    2569 generic.go:356] "Generic (PLEG): container finished" podID="12dea14e-b49c-4a04-9bd9-57bb4b4f5a89" containerID="c0455e605e3ea2673e9999b400512c77869f92c88cd0541326bbdf744f27f33b" exitCode=0
Jul 30 03:48:36.405325 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:36.404994    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6" event={"ID":"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89","Type":"ContainerDied","Data":"c0455e605e3ea2673e9999b400512c77869f92c88cd0541326bbdf744f27f33b"}
Jul 30 03:48:36.996354 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:36.996321    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:37.047316 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:37.047285    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:37.534966 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:37.534945    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"
Jul 30 03:48:37.674724 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:37.674654    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkwp8\" (UniqueName: \"kubernetes.io/projected/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-kube-api-access-bkwp8\") pod \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\" (UID: \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\") "
Jul 30 03:48:37.674724 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:37.674703    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-bundle\") pod \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\" (UID: \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\") "
Jul 30 03:48:37.674958 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:37.674729    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-util\") pod \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\" (UID: \"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89\") "
Jul 30 03:48:37.675402 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:37.675374    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-bundle" (OuterVolumeSpecName: "bundle") pod "12dea14e-b49c-4a04-9bd9-57bb4b4f5a89" (UID: "12dea14e-b49c-4a04-9bd9-57bb4b4f5a89"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:48:37.676937 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:37.676908    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-kube-api-access-bkwp8" (OuterVolumeSpecName: "kube-api-access-bkwp8") pod "12dea14e-b49c-4a04-9bd9-57bb4b4f5a89" (UID: "12dea14e-b49c-4a04-9bd9-57bb4b4f5a89"). InnerVolumeSpecName "kube-api-access-bkwp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:48:37.679325 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:37.679302    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-util" (OuterVolumeSpecName: "util") pod "12dea14e-b49c-4a04-9bd9-57bb4b4f5a89" (UID: "12dea14e-b49c-4a04-9bd9-57bb4b4f5a89"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:48:37.775342 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:37.775308    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkwp8\" (UniqueName: \"kubernetes.io/projected/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-kube-api-access-bkwp8\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:48:37.775342 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:37.775340    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-bundle\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:48:37.775342 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:37.775349    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12dea14e-b49c-4a04-9bd9-57bb4b4f5a89-util\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:48:38.414057 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:38.414015    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6" event={"ID":"12dea14e-b49c-4a04-9bd9-57bb4b4f5a89","Type":"ContainerDied","Data":"277189ad85e0df8c479dfa8f0594cd0b4e89cd906cd17259b385ef42f8e4752f"}
Jul 30 03:48:38.414057 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:38.414037    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402szn6"
Jul 30 03:48:38.414251 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:38.414090    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="277189ad85e0df8c479dfa8f0594cd0b4e89cd906cd17259b385ef42f8e4752f"
Jul 30 03:48:39.535466 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:39.535434    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:40.292226 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:40.292190    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:41.479951 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.479918    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hmtr"]
Jul 30 03:48:41.480423 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.480273    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12dea14e-b49c-4a04-9bd9-57bb4b4f5a89" containerName="extract"
Jul 30 03:48:41.480423 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.480289    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="12dea14e-b49c-4a04-9bd9-57bb4b4f5a89" containerName="extract"
Jul 30 03:48:41.480423 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.480303    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12dea14e-b49c-4a04-9bd9-57bb4b4f5a89" containerName="util"
Jul 30 03:48:41.480423 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.480310    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="12dea14e-b49c-4a04-9bd9-57bb4b4f5a89" containerName="util"
Jul 30 03:48:41.480423 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.480325    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12dea14e-b49c-4a04-9bd9-57bb4b4f5a89" containerName="pull"
Jul 30 03:48:41.480423 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.480332    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="12dea14e-b49c-4a04-9bd9-57bb4b4f5a89" containerName="pull"
Jul 30 03:48:41.480423 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.480392    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="12dea14e-b49c-4a04-9bd9-57bb4b4f5a89" containerName="extract"
Jul 30 03:48:41.511799 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.511778    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hmtr"]
Jul 30 03:48:41.511947 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.511872    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hmtr"
Jul 30 03:48:41.514112 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.514081    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-rvkg5\""
Jul 30 03:48:41.514112 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.514106    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 30 03:48:41.514269 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.514081    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 30 03:48:41.603936 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.603910    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcj6j\" (UniqueName: \"kubernetes.io/projected/d5310e6b-cd93-4696-b56f-9dfb962766be-kube-api-access-mcj6j\") pod \"cert-manager-operator-controller-manager-55c444bf86-2hmtr\" (UID: \"d5310e6b-cd93-4696-b56f-9dfb962766be\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hmtr"
Jul 30 03:48:41.604062 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.603976    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5310e6b-cd93-4696-b56f-9dfb962766be-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2hmtr\" (UID: \"d5310e6b-cd93-4696-b56f-9dfb962766be\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hmtr"
Jul 30 03:48:41.705267 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.705236    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5310e6b-cd93-4696-b56f-9dfb962766be-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2hmtr\" (UID: \"d5310e6b-cd93-4696-b56f-9dfb962766be\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hmtr"
Jul 30 03:48:41.705393 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.705276    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcj6j\" (UniqueName: \"kubernetes.io/projected/d5310e6b-cd93-4696-b56f-9dfb962766be-kube-api-access-mcj6j\") pod \"cert-manager-operator-controller-manager-55c444bf86-2hmtr\" (UID: \"d5310e6b-cd93-4696-b56f-9dfb962766be\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hmtr"
Jul 30 03:48:41.705752 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.705736    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5310e6b-cd93-4696-b56f-9dfb962766be-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2hmtr\" (UID: \"d5310e6b-cd93-4696-b56f-9dfb962766be\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hmtr"
Jul 30 03:48:41.713350 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.713332    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcj6j\" (UniqueName: \"kubernetes.io/projected/d5310e6b-cd93-4696-b56f-9dfb962766be-kube-api-access-mcj6j\") pod \"cert-manager-operator-controller-manager-55c444bf86-2hmtr\" (UID: \"d5310e6b-cd93-4696-b56f-9dfb962766be\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hmtr"
Jul 30 03:48:41.820621 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.820599    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hmtr"
Jul 30 03:48:41.970806 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:41.970782    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hmtr"]
Jul 30 03:48:41.974076 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:48:41.974039    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5310e6b_cd93_4696_b56f_9dfb962766be.slice/crio-bacda8330539ec80d9f8c0187ecad4c43ab6acba55cf6e6a53a0122c6a122462 WatchSource:0}: Error finding container bacda8330539ec80d9f8c0187ecad4c43ab6acba55cf6e6a53a0122c6a122462: Status 404 returned error can't find the container with id bacda8330539ec80d9f8c0187ecad4c43ab6acba55cf6e6a53a0122c6a122462
Jul 30 03:48:42.429697 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:42.429658    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hmtr" event={"ID":"d5310e6b-cd93-4696-b56f-9dfb962766be","Type":"ContainerStarted","Data":"bacda8330539ec80d9f8c0187ecad4c43ab6acba55cf6e6a53a0122c6a122462"}
Jul 30 03:48:43.081734 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:43.081703    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:43.142002 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:43.141972    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:44.440963 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:44.440884    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hmtr" event={"ID":"d5310e6b-cd93-4696-b56f-9dfb962766be","Type":"ContainerStarted","Data":"f6b1d2aaabc2dc3c525843fa2b41c9e62c3b31f5ee22a7e3da22d2d13040e706"}
Jul 30 03:48:44.458978 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:44.458923    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hmtr" podStartSLOduration=1.378596949 podStartE2EDuration="3.458908732s" totalImagesPullingTime="2.080311783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:48:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:48:44.457481853 +0000 UTC m=+223.496192871" watchObservedRunningTime="2026-07-30 03:48:44.458908732 +0000 UTC m=+223.497619751"
Jul 30 03:48:44.832168 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:44.832132    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:44.926666 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:44.926630    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:45.508415 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.508385    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"]
Jul 30 03:48:45.510909 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.510893    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"
Jul 30 03:48:45.512922 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.512897    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 30 03:48:45.513041 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.512921    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dqq9q\""
Jul 30 03:48:45.513524 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.513509    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 30 03:48:45.518743 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.518722    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"]
Jul 30 03:48:45.636038 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.636008    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99wjd\" (UniqueName: \"kubernetes.io/projected/36a51258-7e83-450c-9eb2-858787d0abdb-kube-api-access-99wjd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch\" (UID: \"36a51258-7e83-450c-9eb2-858787d0abdb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"
Jul 30 03:48:45.636168 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.636056    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36a51258-7e83-450c-9eb2-858787d0abdb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch\" (UID: \"36a51258-7e83-450c-9eb2-858787d0abdb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"
Jul 30 03:48:45.636168 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.636077    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36a51258-7e83-450c-9eb2-858787d0abdb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch\" (UID: \"36a51258-7e83-450c-9eb2-858787d0abdb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"
Jul 30 03:48:45.737423 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.737389    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36a51258-7e83-450c-9eb2-858787d0abdb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch\" (UID: \"36a51258-7e83-450c-9eb2-858787d0abdb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"
Jul 30 03:48:45.737580 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.737425    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36a51258-7e83-450c-9eb2-858787d0abdb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch\" (UID: \"36a51258-7e83-450c-9eb2-858787d0abdb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"
Jul 30 03:48:45.737580 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.737504    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99wjd\" (UniqueName: \"kubernetes.io/projected/36a51258-7e83-450c-9eb2-858787d0abdb-kube-api-access-99wjd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch\" (UID: \"36a51258-7e83-450c-9eb2-858787d0abdb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"
Jul 30 03:48:45.737781 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.737761    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36a51258-7e83-450c-9eb2-858787d0abdb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch\" (UID: \"36a51258-7e83-450c-9eb2-858787d0abdb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"
Jul 30 03:48:45.737842 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.737795    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36a51258-7e83-450c-9eb2-858787d0abdb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch\" (UID: \"36a51258-7e83-450c-9eb2-858787d0abdb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"
Jul 30 03:48:45.748037 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.748009    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99wjd\" (UniqueName: \"kubernetes.io/projected/36a51258-7e83-450c-9eb2-858787d0abdb-kube-api-access-99wjd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch\" (UID: \"36a51258-7e83-450c-9eb2-858787d0abdb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"
Jul 30 03:48:45.820945 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.820922    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"
Jul 30 03:48:45.942614 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:45.942586    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"]
Jul 30 03:48:45.945405 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:48:45.945378    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a51258_7e83_450c_9eb2_858787d0abdb.slice/crio-ff3e65a8111a8c49221a5de8a894f15180c42a0bd18b72910b0530c51474be66 WatchSource:0}: Error finding container ff3e65a8111a8c49221a5de8a894f15180c42a0bd18b72910b0530c51474be66: Status 404 returned error can't find the container with id ff3e65a8111a8c49221a5de8a894f15180c42a0bd18b72910b0530c51474be66
Jul 30 03:48:46.449515 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:46.449479    2569 generic.go:356] "Generic (PLEG): container finished" podID="36a51258-7e83-450c-9eb2-858787d0abdb" containerID="b2f2e79aa206d8a3224b82c41589d76043aabded8b45deb7fd5bea2e08825de1" exitCode=0
Jul 30 03:48:46.449690 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:46.449567    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch" event={"ID":"36a51258-7e83-450c-9eb2-858787d0abdb","Type":"ContainerDied","Data":"b2f2e79aa206d8a3224b82c41589d76043aabded8b45deb7fd5bea2e08825de1"}
Jul 30 03:48:46.449690 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:46.449617    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch" event={"ID":"36a51258-7e83-450c-9eb2-858787d0abdb","Type":"ContainerStarted","Data":"ff3e65a8111a8c49221a5de8a894f15180c42a0bd18b72910b0530c51474be66"}
Jul 30 03:48:47.305796 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:47.305752    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:47.494315 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:47.494281    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:48.313979 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:48.313945    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:48.414782 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:48.414751    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:48.462032 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:48.461995    2569 generic.go:356] "Generic (PLEG): container finished" podID="36a51258-7e83-450c-9eb2-858787d0abdb" containerID="ea25aeb38c417dd853743a09440621ca8e0b8061500c08021f3ee529f3a237b4" exitCode=0
Jul 30 03:48:48.462135 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:48.462033    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch" event={"ID":"36a51258-7e83-450c-9eb2-858787d0abdb","Type":"ContainerDied","Data":"ea25aeb38c417dd853743a09440621ca8e0b8061500c08021f3ee529f3a237b4"}
Jul 30 03:48:49.425322 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.425279    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84cc994789-mbnkd" podUID="584e5576-0b0f-450b-b079-eacd575d16c4" containerName="console" containerID="cri-o://afff1af79f8caef39cd105b8e27fa746f301786dd08b63bab3da790856be341b" gracePeriod=15
Jul 30 03:48:49.467663 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.467638    2569 generic.go:356] "Generic (PLEG): container finished" podID="36a51258-7e83-450c-9eb2-858787d0abdb" containerID="1cb7322284cea8cee4d560fa0a6ebd38c850bbc57d26bb1758dc532f04487b05" exitCode=0
Jul 30 03:48:49.467772 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.467715    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch" event={"ID":"36a51258-7e83-450c-9eb2-858787d0abdb","Type":"ContainerDied","Data":"1cb7322284cea8cee4d560fa0a6ebd38c850bbc57d26bb1758dc532f04487b05"}
Jul 30 03:48:49.673712 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.673689    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:48:49.769993 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.769970    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-service-ca\") pod \"584e5576-0b0f-450b-b079-eacd575d16c4\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") "
Jul 30 03:48:49.770125 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.770014    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-trusted-ca-bundle\") pod \"584e5576-0b0f-450b-b079-eacd575d16c4\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") "
Jul 30 03:48:49.770125 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.770041    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-oauth-serving-cert\") pod \"584e5576-0b0f-450b-b079-eacd575d16c4\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") "
Jul 30 03:48:49.770125 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.770071    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-console-config\") pod \"584e5576-0b0f-450b-b079-eacd575d16c4\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") "
Jul 30 03:48:49.770125 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.770091    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ng8ts\" (UniqueName: \"kubernetes.io/projected/584e5576-0b0f-450b-b079-eacd575d16c4-kube-api-access-ng8ts\") pod \"584e5576-0b0f-450b-b079-eacd575d16c4\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") "
Jul 30 03:48:49.770125 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.770123    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/584e5576-0b0f-450b-b079-eacd575d16c4-console-serving-cert\") pod \"584e5576-0b0f-450b-b079-eacd575d16c4\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") "
Jul 30 03:48:49.770348 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.770171    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/584e5576-0b0f-450b-b079-eacd575d16c4-console-oauth-config\") pod \"584e5576-0b0f-450b-b079-eacd575d16c4\" (UID: \"584e5576-0b0f-450b-b079-eacd575d16c4\") "
Jul 30 03:48:49.770348 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.770327    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-service-ca" (OuterVolumeSpecName: "service-ca") pod "584e5576-0b0f-450b-b079-eacd575d16c4" (UID: "584e5576-0b0f-450b-b079-eacd575d16c4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:48:49.770527 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.770503    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "584e5576-0b0f-450b-b079-eacd575d16c4" (UID: "584e5576-0b0f-450b-b079-eacd575d16c4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:48:49.770622 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.770570    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-console-config" (OuterVolumeSpecName: "console-config") pod "584e5576-0b0f-450b-b079-eacd575d16c4" (UID: "584e5576-0b0f-450b-b079-eacd575d16c4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:48:49.770622 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.770591    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "584e5576-0b0f-450b-b079-eacd575d16c4" (UID: "584e5576-0b0f-450b-b079-eacd575d16c4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:48:49.772419 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.772381    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/584e5576-0b0f-450b-b079-eacd575d16c4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "584e5576-0b0f-450b-b079-eacd575d16c4" (UID: "584e5576-0b0f-450b-b079-eacd575d16c4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:48:49.772479 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.772438    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/584e5576-0b0f-450b-b079-eacd575d16c4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "584e5576-0b0f-450b-b079-eacd575d16c4" (UID: "584e5576-0b0f-450b-b079-eacd575d16c4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:48:49.772479 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.772464    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/584e5576-0b0f-450b-b079-eacd575d16c4-kube-api-access-ng8ts" (OuterVolumeSpecName: "kube-api-access-ng8ts") pod "584e5576-0b0f-450b-b079-eacd575d16c4" (UID: "584e5576-0b0f-450b-b079-eacd575d16c4"). InnerVolumeSpecName "kube-api-access-ng8ts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:48:49.870991 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.870962    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-console-config\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:48:49.870991 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.870987    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ng8ts\" (UniqueName: \"kubernetes.io/projected/584e5576-0b0f-450b-b079-eacd575d16c4-kube-api-access-ng8ts\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:48:49.871145 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.871001    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/584e5576-0b0f-450b-b079-eacd575d16c4-console-serving-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:48:49.871145 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.871013    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/584e5576-0b0f-450b-b079-eacd575d16c4-console-oauth-config\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:48:49.871145 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.871025    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:48:49.871145 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.871036    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-trusted-ca-bundle\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:48:49.871145 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.871047    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/584e5576-0b0f-450b-b079-eacd575d16c4-oauth-serving-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:48:49.881323 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.881298    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:49.933962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:49.933938    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:50.473712 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.473682    2569 generic.go:356] "Generic (PLEG): container finished" podID="584e5576-0b0f-450b-b079-eacd575d16c4" containerID="afff1af79f8caef39cd105b8e27fa746f301786dd08b63bab3da790856be341b" exitCode=0
Jul 30 03:48:50.474158 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.473722    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84cc994789-mbnkd"
Jul 30 03:48:50.474158 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.473770    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84cc994789-mbnkd" event={"ID":"584e5576-0b0f-450b-b079-eacd575d16c4","Type":"ContainerDied","Data":"afff1af79f8caef39cd105b8e27fa746f301786dd08b63bab3da790856be341b"}
Jul 30 03:48:50.474158 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.473822    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84cc994789-mbnkd" event={"ID":"584e5576-0b0f-450b-b079-eacd575d16c4","Type":"ContainerDied","Data":"e42315a556d104eb6854cd5cb84fc1a23c6ec0ec89b4fdb65d0a710d5e436a50"}
Jul 30 03:48:50.474158 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.473853    2569 scope.go:117] "RemoveContainer" containerID="afff1af79f8caef39cd105b8e27fa746f301786dd08b63bab3da790856be341b"
Jul 30 03:48:50.482849 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.482829    2569 scope.go:117] "RemoveContainer" containerID="afff1af79f8caef39cd105b8e27fa746f301786dd08b63bab3da790856be341b"
Jul 30 03:48:50.483138 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:48:50.483119    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afff1af79f8caef39cd105b8e27fa746f301786dd08b63bab3da790856be341b\": container with ID starting with afff1af79f8caef39cd105b8e27fa746f301786dd08b63bab3da790856be341b not found: ID does not exist" containerID="afff1af79f8caef39cd105b8e27fa746f301786dd08b63bab3da790856be341b"
Jul 30 03:48:50.483193 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.483147    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afff1af79f8caef39cd105b8e27fa746f301786dd08b63bab3da790856be341b"} err="failed to get container status \"afff1af79f8caef39cd105b8e27fa746f301786dd08b63bab3da790856be341b\": rpc error: code = NotFound desc = could not find container \"afff1af79f8caef39cd105b8e27fa746f301786dd08b63bab3da790856be341b\": container with ID starting with afff1af79f8caef39cd105b8e27fa746f301786dd08b63bab3da790856be341b not found: ID does not exist"
Jul 30 03:48:50.497084 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.497061    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84cc994789-mbnkd"]
Jul 30 03:48:50.507468 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.507444    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84cc994789-mbnkd"]
Jul 30 03:48:50.612505 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.612479    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"
Jul 30 03:48:50.784394 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.784367    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36a51258-7e83-450c-9eb2-858787d0abdb-bundle\") pod \"36a51258-7e83-450c-9eb2-858787d0abdb\" (UID: \"36a51258-7e83-450c-9eb2-858787d0abdb\") "
Jul 30 03:48:50.784593 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.784438    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36a51258-7e83-450c-9eb2-858787d0abdb-util\") pod \"36a51258-7e83-450c-9eb2-858787d0abdb\" (UID: \"36a51258-7e83-450c-9eb2-858787d0abdb\") "
Jul 30 03:48:50.784593 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.784474    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99wjd\" (UniqueName: \"kubernetes.io/projected/36a51258-7e83-450c-9eb2-858787d0abdb-kube-api-access-99wjd\") pod \"36a51258-7e83-450c-9eb2-858787d0abdb\" (UID: \"36a51258-7e83-450c-9eb2-858787d0abdb\") "
Jul 30 03:48:50.784813 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.784785    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36a51258-7e83-450c-9eb2-858787d0abdb-bundle" (OuterVolumeSpecName: "bundle") pod "36a51258-7e83-450c-9eb2-858787d0abdb" (UID: "36a51258-7e83-450c-9eb2-858787d0abdb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:48:50.786611 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.786589    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36a51258-7e83-450c-9eb2-858787d0abdb-kube-api-access-99wjd" (OuterVolumeSpecName: "kube-api-access-99wjd") pod "36a51258-7e83-450c-9eb2-858787d0abdb" (UID: "36a51258-7e83-450c-9eb2-858787d0abdb"). InnerVolumeSpecName "kube-api-access-99wjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:48:50.789860 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.789838    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36a51258-7e83-450c-9eb2-858787d0abdb-util" (OuterVolumeSpecName: "util") pod "36a51258-7e83-450c-9eb2-858787d0abdb" (UID: "36a51258-7e83-450c-9eb2-858787d0abdb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:48:50.885230 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.885206    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36a51258-7e83-450c-9eb2-858787d0abdb-util\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:48:50.885230 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.885228    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99wjd\" (UniqueName: \"kubernetes.io/projected/36a51258-7e83-450c-9eb2-858787d0abdb-kube-api-access-99wjd\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:48:50.885368 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:50.885238    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36a51258-7e83-450c-9eb2-858787d0abdb-bundle\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:48:51.464132 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:51.464102    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="584e5576-0b0f-450b-b079-eacd575d16c4" path="/var/lib/kubelet/pods/584e5576-0b0f-450b-b079-eacd575d16c4/volumes"
Jul 30 03:48:51.478701 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:51.478674    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch" event={"ID":"36a51258-7e83-450c-9eb2-858787d0abdb","Type":"ContainerDied","Data":"ff3e65a8111a8c49221a5de8a894f15180c42a0bd18b72910b0530c51474be66"}
Jul 30 03:48:51.479087 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:51.478720    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff3e65a8111a8c49221a5de8a894f15180c42a0bd18b72910b0530c51474be66"
Jul 30 03:48:51.479087 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:51.478681    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcvxch"
Jul 30 03:48:54.927582 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:54.927553    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:55.740989 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.740955    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dfkk4"]
Jul 30 03:48:55.741281 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.741264    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="584e5576-0b0f-450b-b079-eacd575d16c4" containerName="console"
Jul 30 03:48:55.741361 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.741284    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="584e5576-0b0f-450b-b079-eacd575d16c4" containerName="console"
Jul 30 03:48:55.741361 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.741306    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a51258-7e83-450c-9eb2-858787d0abdb" containerName="extract"
Jul 30 03:48:55.741361 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.741314    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a51258-7e83-450c-9eb2-858787d0abdb" containerName="extract"
Jul 30 03:48:55.741361 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.741326    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a51258-7e83-450c-9eb2-858787d0abdb" containerName="util"
Jul 30 03:48:55.741361 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.741335    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a51258-7e83-450c-9eb2-858787d0abdb" containerName="util"
Jul 30 03:48:55.741361 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.741357    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a51258-7e83-450c-9eb2-858787d0abdb" containerName="pull"
Jul 30 03:48:55.741687 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.741364    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a51258-7e83-450c-9eb2-858787d0abdb" containerName="pull"
Jul 30 03:48:55.741687 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.741432    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="36a51258-7e83-450c-9eb2-858787d0abdb" containerName="extract"
Jul 30 03:48:55.741687 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.741447    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="584e5576-0b0f-450b-b079-eacd575d16c4" containerName="console"
Jul 30 03:48:55.744465 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.744445    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-dfkk4"
Jul 30 03:48:55.746544 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.746504    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 30 03:48:55.746652 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.746624    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 30 03:48:55.747198 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.747180    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7rkvq\""
Jul 30 03:48:55.753920 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.753895    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dfkk4"]
Jul 30 03:48:55.915635 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.915605    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-824xr\" (UniqueName: \"kubernetes.io/projected/c8482eef-7a2d-4b55-9d01-2ed7510af788-kube-api-access-824xr\") pod \"cert-manager-55b4d954d5-dfkk4\" (UID: \"c8482eef-7a2d-4b55-9d01-2ed7510af788\") " pod="cert-manager/cert-manager-55b4d954d5-dfkk4"
Jul 30 03:48:55.915635 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:55.915641    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c8482eef-7a2d-4b55-9d01-2ed7510af788-bound-sa-token\") pod \"cert-manager-55b4d954d5-dfkk4\" (UID: \"c8482eef-7a2d-4b55-9d01-2ed7510af788\") " pod="cert-manager/cert-manager-55b4d954d5-dfkk4"
Jul 30 03:48:56.016198 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:56.016167    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-824xr\" (UniqueName: \"kubernetes.io/projected/c8482eef-7a2d-4b55-9d01-2ed7510af788-kube-api-access-824xr\") pod \"cert-manager-55b4d954d5-dfkk4\" (UID: \"c8482eef-7a2d-4b55-9d01-2ed7510af788\") " pod="cert-manager/cert-manager-55b4d954d5-dfkk4"
Jul 30 03:48:56.016597 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:56.016202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c8482eef-7a2d-4b55-9d01-2ed7510af788-bound-sa-token\") pod \"cert-manager-55b4d954d5-dfkk4\" (UID: \"c8482eef-7a2d-4b55-9d01-2ed7510af788\") " pod="cert-manager/cert-manager-55b4d954d5-dfkk4"
Jul 30 03:48:56.023618 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:56.023595    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c8482eef-7a2d-4b55-9d01-2ed7510af788-bound-sa-token\") pod \"cert-manager-55b4d954d5-dfkk4\" (UID: \"c8482eef-7a2d-4b55-9d01-2ed7510af788\") " pod="cert-manager/cert-manager-55b4d954d5-dfkk4"
Jul 30 03:48:56.023696 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:56.023655    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-824xr\" (UniqueName: \"kubernetes.io/projected/c8482eef-7a2d-4b55-9d01-2ed7510af788-kube-api-access-824xr\") pod \"cert-manager-55b4d954d5-dfkk4\" (UID: \"c8482eef-7a2d-4b55-9d01-2ed7510af788\") " pod="cert-manager/cert-manager-55b4d954d5-dfkk4"
Jul 30 03:48:56.053406 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:56.053381    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-dfkk4"
Jul 30 03:48:56.178283 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:56.178245    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dfkk4"]
Jul 30 03:48:56.181761 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:48:56.181737    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8482eef_7a2d_4b55_9d01_2ed7510af788.slice/crio-f43bcb2ba3e3fdebb24e03d6163e03b9b7d1908b4a22a2ea78480520aa601788 WatchSource:0}: Error finding container f43bcb2ba3e3fdebb24e03d6163e03b9b7d1908b4a22a2ea78480520aa601788: Status 404 returned error can't find the container with id f43bcb2ba3e3fdebb24e03d6163e03b9b7d1908b4a22a2ea78480520aa601788
Jul 30 03:48:56.490685 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:56.490607    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:56.499521 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:56.499491    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-dfkk4" event={"ID":"c8482eef-7a2d-4b55-9d01-2ed7510af788","Type":"ContainerStarted","Data":"f43bcb2ba3e3fdebb24e03d6163e03b9b7d1908b4a22a2ea78480520aa601788"}
Jul 30 03:48:56.590873 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:56.590835    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:58.507766 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.507728    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:58.554761 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.554736    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mf6zs"]
Jul 30 03:48:58.559060 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.559042    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mf6zs"
Jul 30 03:48:58.561099 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.561081    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 30 03:48:58.561182 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.561081    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 30 03:48:58.561636 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.561620    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-nj4lt\""
Jul 30 03:48:58.565977 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.565957    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mf6zs"]
Jul 30 03:48:58.578330 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.578308    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:58.633550 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.633509    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb619dd6-621f-47c4-a6b1-4d7fb30bbac7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mf6zs\" (UID: \"bb619dd6-621f-47c4-a6b1-4d7fb30bbac7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mf6zs"
Jul 30 03:48:58.633675 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.633569    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlmnl\" (UniqueName: \"kubernetes.io/projected/bb619dd6-621f-47c4-a6b1-4d7fb30bbac7-kube-api-access-nlmnl\") pod \"openshift-lws-operator-fd8ccff4c-mf6zs\" (UID: \"bb619dd6-621f-47c4-a6b1-4d7fb30bbac7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mf6zs"
Jul 30 03:48:58.734588 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.734557    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb619dd6-621f-47c4-a6b1-4d7fb30bbac7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mf6zs\" (UID: \"bb619dd6-621f-47c4-a6b1-4d7fb30bbac7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mf6zs"
Jul 30 03:48:58.734764 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.734607    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlmnl\" (UniqueName: \"kubernetes.io/projected/bb619dd6-621f-47c4-a6b1-4d7fb30bbac7-kube-api-access-nlmnl\") pod \"openshift-lws-operator-fd8ccff4c-mf6zs\" (UID: \"bb619dd6-621f-47c4-a6b1-4d7fb30bbac7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mf6zs"
Jul 30 03:48:58.734937 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.734915    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb619dd6-621f-47c4-a6b1-4d7fb30bbac7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mf6zs\" (UID: \"bb619dd6-621f-47c4-a6b1-4d7fb30bbac7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mf6zs"
Jul 30 03:48:58.742035 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.742008    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlmnl\" (UniqueName: \"kubernetes.io/projected/bb619dd6-621f-47c4-a6b1-4d7fb30bbac7-kube-api-access-nlmnl\") pod \"openshift-lws-operator-fd8ccff4c-mf6zs\" (UID: \"bb619dd6-621f-47c4-a6b1-4d7fb30bbac7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mf6zs"
Jul 30 03:48:58.867966 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.867929    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mf6zs"
Jul 30 03:48:58.990693 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:58.990662    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mf6zs"]
Jul 30 03:48:58.993482 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:48:58.993455    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb619dd6_621f_47c4_a6b1_4d7fb30bbac7.slice/crio-8a5895d5e7077228d59bf3ddc6859adb7d371f5f9f0c987bcaf3a8c0d2fcdbbf WatchSource:0}: Error finding container 8a5895d5e7077228d59bf3ddc6859adb7d371f5f9f0c987bcaf3a8c0d2fcdbbf: Status 404 returned error can't find the container with id 8a5895d5e7077228d59bf3ddc6859adb7d371f5f9f0c987bcaf3a8c0d2fcdbbf
Jul 30 03:48:59.513549 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:59.513489    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-dfkk4" event={"ID":"c8482eef-7a2d-4b55-9d01-2ed7510af788","Type":"ContainerStarted","Data":"be5d907fa31334b4b456ba749301ccfff2fb87a8a3bfab8a48eb85c6002ca77b"}
Jul 30 03:48:59.515025 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:59.514990    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mf6zs" event={"ID":"bb619dd6-621f-47c4-a6b1-4d7fb30bbac7","Type":"ContainerStarted","Data":"8a5895d5e7077228d59bf3ddc6859adb7d371f5f9f0c987bcaf3a8c0d2fcdbbf"}
Jul 30 03:48:59.528221 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:48:59.528179    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-dfkk4" podStartSLOduration=2.193267067 podStartE2EDuration="4.528161189s" totalImagesPullingTime="2.334894122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:48:59.527549784 +0000 UTC m=+238.566260793" watchObservedRunningTime="2026-07-30 03:48:59.528161189 +0000 UTC m=+238.566872207"
Jul 30 03:49:01.524578 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:01.524522    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mf6zs" event={"ID":"bb619dd6-621f-47c4-a6b1-4d7fb30bbac7","Type":"ContainerStarted","Data":"e25d0dfe889a3db1cadd7fc8b8bd176f0bc36efa443697daeefba577c832ec58"}
Jul 30 03:49:01.539403 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:01.539356    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mf6zs" podStartSLOduration=2.002420939 podStartE2EDuration="3.539341406s" totalImagesPullingTime="1.536920467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:49:01.537643802 +0000 UTC m=+240.576354821" watchObservedRunningTime="2026-07-30 03:49:01.539341406 +0000 UTC m=+240.578052424"
Jul 30 03:49:02.327477 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:02.327441    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:02.431195 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:02.431147    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:04.373979 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:04.373944    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:06.359987 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.359960    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"]
Jul 30 03:49:06.385070 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.385048    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"]
Jul 30 03:49:06.385221 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.385142    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"
Jul 30 03:49:06.387266 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.387244    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dqq9q\""
Jul 30 03:49:06.387266 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.387256    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 30 03:49:06.387852 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.387833    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 30 03:49:06.484478 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.484450    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45b2a8c5-9152-43ee-9df1-cc717c259fa0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc\" (UID: \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"
Jul 30 03:49:06.484620 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.484505    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5z67\" (UniqueName: \"kubernetes.io/projected/45b2a8c5-9152-43ee-9df1-cc717c259fa0-kube-api-access-z5z67\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc\" (UID: \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"
Jul 30 03:49:06.484620 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.484571    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45b2a8c5-9152-43ee-9df1-cc717c259fa0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc\" (UID: \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"
Jul 30 03:49:06.585300 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.585272    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45b2a8c5-9152-43ee-9df1-cc717c259fa0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc\" (UID: \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"
Jul 30 03:49:06.585410 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.585324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45b2a8c5-9152-43ee-9df1-cc717c259fa0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc\" (UID: \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"
Jul 30 03:49:06.585410 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.585364    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5z67\" (UniqueName: \"kubernetes.io/projected/45b2a8c5-9152-43ee-9df1-cc717c259fa0-kube-api-access-z5z67\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc\" (UID: \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"
Jul 30 03:49:06.585702 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.585679    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45b2a8c5-9152-43ee-9df1-cc717c259fa0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc\" (UID: \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"
Jul 30 03:49:06.585702 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.585691    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45b2a8c5-9152-43ee-9df1-cc717c259fa0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc\" (UID: \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"
Jul 30 03:49:06.593020 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.593001    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5z67\" (UniqueName: \"kubernetes.io/projected/45b2a8c5-9152-43ee-9df1-cc717c259fa0-kube-api-access-z5z67\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc\" (UID: \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"
Jul 30 03:49:06.694770 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.694707    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"
Jul 30 03:49:06.842138 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:06.842107    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"]
Jul 30 03:49:06.844515 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:49:06.844488    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45b2a8c5_9152_43ee_9df1_cc717c259fa0.slice/crio-36725370ccbdbc2c9f6e8bc6b16245cee37dff67a8ee9ced02e71d5081b45579 WatchSource:0}: Error finding container 36725370ccbdbc2c9f6e8bc6b16245cee37dff67a8ee9ced02e71d5081b45579: Status 404 returned error can't find the container with id 36725370ccbdbc2c9f6e8bc6b16245cee37dff67a8ee9ced02e71d5081b45579
Jul 30 03:49:07.548654 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:07.548619    2569 generic.go:356] "Generic (PLEG): container finished" podID="45b2a8c5-9152-43ee-9df1-cc717c259fa0" containerID="45b08faa7a22db1d8fdd31008acf64e8c6602152a16897a592851c9f53fd1f1d" exitCode=0
Jul 30 03:49:07.549039 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:07.548692    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc" event={"ID":"45b2a8c5-9152-43ee-9df1-cc717c259fa0","Type":"ContainerDied","Data":"45b08faa7a22db1d8fdd31008acf64e8c6602152a16897a592851c9f53fd1f1d"}
Jul 30 03:49:07.549039 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:07.548738    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc" event={"ID":"45b2a8c5-9152-43ee-9df1-cc717c259fa0","Type":"ContainerStarted","Data":"36725370ccbdbc2c9f6e8bc6b16245cee37dff67a8ee9ced02e71d5081b45579"}
Jul 30 03:49:08.554717 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:08.554685    2569 generic.go:356] "Generic (PLEG): container finished" podID="45b2a8c5-9152-43ee-9df1-cc717c259fa0" containerID="0722a30c07813ba4870994a85da44cd3c63e209cf660e740c5223e9ecc42421a" exitCode=0
Jul 30 03:49:08.555077 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:08.554749    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc" event={"ID":"45b2a8c5-9152-43ee-9df1-cc717c259fa0","Type":"ContainerDied","Data":"0722a30c07813ba4870994a85da44cd3c63e209cf660e740c5223e9ecc42421a"}
Jul 30 03:49:09.560827 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:09.560792    2569 generic.go:356] "Generic (PLEG): container finished" podID="45b2a8c5-9152-43ee-9df1-cc717c259fa0" containerID="6a779d622c77aec23b9b319eaf0eb971e5fdc07ac43ccc8621aab6e690f38b5b" exitCode=0
Jul 30 03:49:09.561229 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:09.560865    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc" event={"ID":"45b2a8c5-9152-43ee-9df1-cc717c259fa0","Type":"ContainerDied","Data":"6a779d622c77aec23b9b319eaf0eb971e5fdc07ac43ccc8621aab6e690f38b5b"}
Jul 30 03:49:09.782345 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:09.782308    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:10.688806 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:10.688781    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"
Jul 30 03:49:10.711853 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:10.711827    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5z67\" (UniqueName: \"kubernetes.io/projected/45b2a8c5-9152-43ee-9df1-cc717c259fa0-kube-api-access-z5z67\") pod \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\" (UID: \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\") "
Jul 30 03:49:10.712001 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:10.711882    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45b2a8c5-9152-43ee-9df1-cc717c259fa0-bundle\") pod \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\" (UID: \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\") "
Jul 30 03:49:10.712001 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:10.711922    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45b2a8c5-9152-43ee-9df1-cc717c259fa0-util\") pod \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\" (UID: \"45b2a8c5-9152-43ee-9df1-cc717c259fa0\") "
Jul 30 03:49:10.712568 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:10.712527    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45b2a8c5-9152-43ee-9df1-cc717c259fa0-bundle" (OuterVolumeSpecName: "bundle") pod "45b2a8c5-9152-43ee-9df1-cc717c259fa0" (UID: "45b2a8c5-9152-43ee-9df1-cc717c259fa0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:49:10.713958 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:10.713936    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45b2a8c5-9152-43ee-9df1-cc717c259fa0-kube-api-access-z5z67" (OuterVolumeSpecName: "kube-api-access-z5z67") pod "45b2a8c5-9152-43ee-9df1-cc717c259fa0" (UID: "45b2a8c5-9152-43ee-9df1-cc717c259fa0"). InnerVolumeSpecName "kube-api-access-z5z67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:49:10.717440 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:10.717406    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45b2a8c5-9152-43ee-9df1-cc717c259fa0-util" (OuterVolumeSpecName: "util") pod "45b2a8c5-9152-43ee-9df1-cc717c259fa0" (UID: "45b2a8c5-9152-43ee-9df1-cc717c259fa0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:49:10.812470 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:10.812445    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5z67\" (UniqueName: \"kubernetes.io/projected/45b2a8c5-9152-43ee-9df1-cc717c259fa0-kube-api-access-z5z67\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:49:10.812470 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:10.812469    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45b2a8c5-9152-43ee-9df1-cc717c259fa0-bundle\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:49:10.812631 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:10.812478    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45b2a8c5-9152-43ee-9df1-cc717c259fa0-util\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:49:11.571388 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:11.571352    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc" event={"ID":"45b2a8c5-9152-43ee-9df1-cc717c259fa0","Type":"ContainerDied","Data":"36725370ccbdbc2c9f6e8bc6b16245cee37dff67a8ee9ced02e71d5081b45579"}
Jul 30 03:49:11.571519 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:11.571403    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36725370ccbdbc2c9f6e8bc6b16245cee37dff67a8ee9ced02e71d5081b45579"
Jul 30 03:49:11.571519 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:11.571373    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft9gmc"
Jul 30 03:49:18.468719 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.468685    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:18.611800 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.611754    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:18.738104 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.738020    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:18.745105 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.745077    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"]
Jul 30 03:49:18.745347 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.745335    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45b2a8c5-9152-43ee-9df1-cc717c259fa0" containerName="util"
Jul 30 03:49:18.745392 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.745351    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="45b2a8c5-9152-43ee-9df1-cc717c259fa0" containerName="util"
Jul 30 03:49:18.745392 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.745363    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45b2a8c5-9152-43ee-9df1-cc717c259fa0" containerName="extract"
Jul 30 03:49:18.745392 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.745368    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="45b2a8c5-9152-43ee-9df1-cc717c259fa0" containerName="extract"
Jul 30 03:49:18.745392 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.745379    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45b2a8c5-9152-43ee-9df1-cc717c259fa0" containerName="pull"
Jul 30 03:49:18.745392 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.745384    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="45b2a8c5-9152-43ee-9df1-cc717c259fa0" containerName="pull"
Jul 30 03:49:18.745605 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.745428    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="45b2a8c5-9152-43ee-9df1-cc717c259fa0" containerName="extract"
Jul 30 03:49:18.751230 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.751213    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:18.753552 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.753519    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mhvrh\""
Jul 30 03:49:18.754148 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.754132    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 30 03:49:18.754234 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.754184    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 30 03:49:18.754455 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.754440    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 30 03:49:18.758380 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.758361    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"]
Jul 30 03:49:18.762177 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.762155    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0e2678d-da59-4b4e-99df-f0059e7f1f56-metrics-cert\") pod \"lws-controller-manager-69d4f5b96d-vm9rl\" (UID: \"e0e2678d-da59-4b4e-99df-f0059e7f1f56\") " pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:18.762276 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.762195    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e0e2678d-da59-4b4e-99df-f0059e7f1f56-manager-config\") pod \"lws-controller-manager-69d4f5b96d-vm9rl\" (UID: \"e0e2678d-da59-4b4e-99df-f0059e7f1f56\") " pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:18.762276 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.762247    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv5nr\" (UniqueName: \"kubernetes.io/projected/e0e2678d-da59-4b4e-99df-f0059e7f1f56-kube-api-access-mv5nr\") pod \"lws-controller-manager-69d4f5b96d-vm9rl\" (UID: \"e0e2678d-da59-4b4e-99df-f0059e7f1f56\") " pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:18.762382 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.762284    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0e2678d-da59-4b4e-99df-f0059e7f1f56-cert\") pod \"lws-controller-manager-69d4f5b96d-vm9rl\" (UID: \"e0e2678d-da59-4b4e-99df-f0059e7f1f56\") " pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:18.862643 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.862610    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv5nr\" (UniqueName: \"kubernetes.io/projected/e0e2678d-da59-4b4e-99df-f0059e7f1f56-kube-api-access-mv5nr\") pod \"lws-controller-manager-69d4f5b96d-vm9rl\" (UID: \"e0e2678d-da59-4b4e-99df-f0059e7f1f56\") " pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:18.862803 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.862653    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0e2678d-da59-4b4e-99df-f0059e7f1f56-cert\") pod \"lws-controller-manager-69d4f5b96d-vm9rl\" (UID: \"e0e2678d-da59-4b4e-99df-f0059e7f1f56\") " pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:18.862803 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.862693    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0e2678d-da59-4b4e-99df-f0059e7f1f56-metrics-cert\") pod \"lws-controller-manager-69d4f5b96d-vm9rl\" (UID: \"e0e2678d-da59-4b4e-99df-f0059e7f1f56\") " pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:18.862803 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.862715    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e0e2678d-da59-4b4e-99df-f0059e7f1f56-manager-config\") pod \"lws-controller-manager-69d4f5b96d-vm9rl\" (UID: \"e0e2678d-da59-4b4e-99df-f0059e7f1f56\") " pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:18.863409 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.863382    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e0e2678d-da59-4b4e-99df-f0059e7f1f56-manager-config\") pod \"lws-controller-manager-69d4f5b96d-vm9rl\" (UID: \"e0e2678d-da59-4b4e-99df-f0059e7f1f56\") " pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:18.865403 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.865380    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0e2678d-da59-4b4e-99df-f0059e7f1f56-metrics-cert\") pod \"lws-controller-manager-69d4f5b96d-vm9rl\" (UID: \"e0e2678d-da59-4b4e-99df-f0059e7f1f56\") " pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:18.865498 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.865384    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0e2678d-da59-4b4e-99df-f0059e7f1f56-cert\") pod \"lws-controller-manager-69d4f5b96d-vm9rl\" (UID: \"e0e2678d-da59-4b4e-99df-f0059e7f1f56\") " pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:18.870196 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:18.870171    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv5nr\" (UniqueName: \"kubernetes.io/projected/e0e2678d-da59-4b4e-99df-f0059e7f1f56-kube-api-access-mv5nr\") pod \"lws-controller-manager-69d4f5b96d-vm9rl\" (UID: \"e0e2678d-da59-4b4e-99df-f0059e7f1f56\") " pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:19.061029 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.061000    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:19.190066 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.190028    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"]
Jul 30 03:49:19.193796 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:49:19.193769    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e2678d_da59_4b4e_99df_f0059e7f1f56.slice/crio-c54ffd8215cef0186416dc7668d012bf3b40a051e6ae5f6c55c25a787ab451ef WatchSource:0}: Error finding container c54ffd8215cef0186416dc7668d012bf3b40a051e6ae5f6c55c25a787ab451ef: Status 404 returned error can't find the container with id c54ffd8215cef0186416dc7668d012bf3b40a051e6ae5f6c55c25a787ab451ef
Jul 30 03:49:19.420320 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.420238    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:19.609296 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.609262    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl" event={"ID":"e0e2678d-da59-4b4e-99df-f0059e7f1f56","Type":"ContainerStarted","Data":"c54ffd8215cef0186416dc7668d012bf3b40a051e6ae5f6c55c25a787ab451ef"}
Jul 30 03:49:19.712676 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.712592    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:19.806278 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.806240    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:19.840773 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.840748    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"]
Jul 30 03:49:19.844646 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.844620    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.852653 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.852477    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 03:49:19.852653 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.852506    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 03:49:19.852653 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.852619    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 03:49:19.852653 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.852636    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 03:49:19.853223 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.852947    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mj4sc\""
Jul 30 03:49:19.858689 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.858668    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"]
Jul 30 03:49:19.870105 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.870069    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2eed90ee-243a-45b0-8a41-b13a62f27d0f-webhook-cert\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.870216 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.870162    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2eed90ee-243a-45b0-8a41-b13a62f27d0f-manifests\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.870216 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.870203    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2eed90ee-243a-45b0-8a41-b13a62f27d0f-tmp\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.870324 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.870239    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2eed90ee-243a-45b0-8a41-b13a62f27d0f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.870324 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.870286    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwkmq\" (UniqueName: \"kubernetes.io/projected/2eed90ee-243a-45b0-8a41-b13a62f27d0f-kube-api-access-vwkmq\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.961151 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.961119    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:19.971423 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.971275    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2eed90ee-243a-45b0-8a41-b13a62f27d0f-webhook-cert\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.971423 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.971317    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2eed90ee-243a-45b0-8a41-b13a62f27d0f-manifests\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.971423 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.971356    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2eed90ee-243a-45b0-8a41-b13a62f27d0f-tmp\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.971423 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.971405    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2eed90ee-243a-45b0-8a41-b13a62f27d0f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.971423 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.971432    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwkmq\" (UniqueName: \"kubernetes.io/projected/2eed90ee-243a-45b0-8a41-b13a62f27d0f-kube-api-access-vwkmq\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.971886 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.971783    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2eed90ee-243a-45b0-8a41-b13a62f27d0f-manifests\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.974611 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.974593    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2eed90ee-243a-45b0-8a41-b13a62f27d0f-webhook-cert\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.974692 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.974654    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2eed90ee-243a-45b0-8a41-b13a62f27d0f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.975003 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.974984    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2eed90ee-243a-45b0-8a41-b13a62f27d0f-tmp\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:19.979640 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:19.979619    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwkmq\" (UniqueName: \"kubernetes.io/projected/2eed90ee-243a-45b0-8a41-b13a62f27d0f-kube-api-access-vwkmq\") pod \"opendatahub-operator-controller-manager-599789d45f-g7rh7\" (UID: \"2eed90ee-243a-45b0-8a41-b13a62f27d0f\") " pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:20.065661 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:20.065625    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:20.157129 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:20.157099    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:20.166230 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:20.166202    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:20.266322 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:20.266267    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:20.317461 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:20.317432    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"]
Jul 30 03:49:20.372731 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:20.372643    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:20.590867 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:49:20.590831    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eed90ee_243a_45b0_8a41_b13a62f27d0f.slice/crio-46a867d38d648bd51f3d9e1da881d086d0222da3afa4a39b3cf6d0a11abce821 WatchSource:0}: Error finding container 46a867d38d648bd51f3d9e1da881d086d0222da3afa4a39b3cf6d0a11abce821: Status 404 returned error can't find the container with id 46a867d38d648bd51f3d9e1da881d086d0222da3afa4a39b3cf6d0a11abce821
Jul 30 03:49:20.615011 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:20.614984    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7" event={"ID":"2eed90ee-243a-45b0-8a41-b13a62f27d0f","Type":"ContainerStarted","Data":"46a867d38d648bd51f3d9e1da881d086d0222da3afa4a39b3cf6d0a11abce821"}
Jul 30 03:49:21.623237 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:21.623194    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl" event={"ID":"e0e2678d-da59-4b4e-99df-f0059e7f1f56","Type":"ContainerStarted","Data":"ecf54fb98098a4f013123875eca7aa36f8a0fc5ddb6b93bf98e429895c4b5e17"}
Jul 30 03:49:21.623826 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:21.623297    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:21.643855 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:21.643807    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl" podStartSLOduration=2.186019075 podStartE2EDuration="3.643787989s" totalImagesPullingTime="1.457768914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:49:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:49:21.64134604 +0000 UTC m=+260.680057061" watchObservedRunningTime="2026-07-30 03:49:21.643787989 +0000 UTC m=+260.682499011"
Jul 30 03:49:23.634506 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:23.634462    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7" event={"ID":"2eed90ee-243a-45b0-8a41-b13a62f27d0f","Type":"ContainerStarted","Data":"26e9273b7a54f9793c10dd20c195ea09bb2463f51d2a143efa49237f281075cf"}
Jul 30 03:49:24.640185 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:24.640154    2569 generic.go:356] "Generic (PLEG): container finished" podID="2eed90ee-243a-45b0-8a41-b13a62f27d0f" containerID="26e9273b7a54f9793c10dd20c195ea09bb2463f51d2a143efa49237f281075cf" exitCode=0
Jul 30 03:49:24.640515 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:24.640219    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7" event={"ID":"2eed90ee-243a-45b0-8a41-b13a62f27d0f","Type":"ContainerDied","Data":"26e9273b7a54f9793c10dd20c195ea09bb2463f51d2a143efa49237f281075cf"}
Jul 30 03:49:25.645991 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:25.645958    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7" event={"ID":"2eed90ee-243a-45b0-8a41-b13a62f27d0f","Type":"ContainerStarted","Data":"f017dbc1f0f033501faea58082e11cfd54f2faa984091867d8014fec9a0868f2"}
Jul 30 03:49:25.646374 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:25.646095    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:25.666892 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:25.666837    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7" podStartSLOduration=2.535136299 podStartE2EDuration="6.666819183s" totalImagesPullingTime="3.137620007s" totalInitContainerRuntime="994.062877ms" isStatefulPod=false podCreationTimestamp="2026-07-30 03:49:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:49:25.664998224 +0000 UTC m=+264.703709245" watchObservedRunningTime="2026-07-30 03:49:25.666819183 +0000 UTC m=+264.705530203"
Jul 30 03:49:32.630262 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:32.630229    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-69d4f5b96d-vm9rl"
Jul 30 03:49:36.652485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:36.652455    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-599789d45f-g7rh7"
Jul 30 03:49:39.590739 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:39.590703    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:39.704083 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:39.704050    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:40.106283 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.106232    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:40.192312 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.192280    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-58ccc58984-75hc6"]
Jul 30 03:49:40.202650 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.202627    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58ccc58984-75hc6"
Jul 30 03:49:40.205095 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.205067    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 30 03:49:40.205255 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.205189    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 03:49:40.205602 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.205585    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 30 03:49:40.205880 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.205862    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 03:49:40.206214 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.206176    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qhd9l\""
Jul 30 03:49:40.207150 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.207129    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58ccc58984-75hc6"]
Jul 30 03:49:40.248050 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.248014    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:40.320803 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.320770    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dd4c125-2623-4e9e-a868-f4ce0157d66a-tmp\") pod \"kube-auth-proxy-58ccc58984-75hc6\" (UID: \"8dd4c125-2623-4e9e-a868-f4ce0157d66a\") " pod="openshift-ingress/kube-auth-proxy-58ccc58984-75hc6"
Jul 30 03:49:40.320933 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.320814    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd4c125-2623-4e9e-a868-f4ce0157d66a-tls-certs\") pod \"kube-auth-proxy-58ccc58984-75hc6\" (UID: \"8dd4c125-2623-4e9e-a868-f4ce0157d66a\") " pod="openshift-ingress/kube-auth-proxy-58ccc58984-75hc6"
Jul 30 03:49:40.320933 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.320880    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j5hf\" (UniqueName: \"kubernetes.io/projected/8dd4c125-2623-4e9e-a868-f4ce0157d66a-kube-api-access-5j5hf\") pod \"kube-auth-proxy-58ccc58984-75hc6\" (UID: \"8dd4c125-2623-4e9e-a868-f4ce0157d66a\") " pod="openshift-ingress/kube-auth-proxy-58ccc58984-75hc6"
Jul 30 03:49:40.389067 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.388989    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:40.421821 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.421788    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dd4c125-2623-4e9e-a868-f4ce0157d66a-tmp\") pod \"kube-auth-proxy-58ccc58984-75hc6\" (UID: \"8dd4c125-2623-4e9e-a868-f4ce0157d66a\") " pod="openshift-ingress/kube-auth-proxy-58ccc58984-75hc6"
Jul 30 03:49:40.421984 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.421862    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd4c125-2623-4e9e-a868-f4ce0157d66a-tls-certs\") pod \"kube-auth-proxy-58ccc58984-75hc6\" (UID: \"8dd4c125-2623-4e9e-a868-f4ce0157d66a\") " pod="openshift-ingress/kube-auth-proxy-58ccc58984-75hc6"
Jul 30 03:49:40.421984 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.421909    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j5hf\" (UniqueName: \"kubernetes.io/projected/8dd4c125-2623-4e9e-a868-f4ce0157d66a-kube-api-access-5j5hf\") pod \"kube-auth-proxy-58ccc58984-75hc6\" (UID: \"8dd4c125-2623-4e9e-a868-f4ce0157d66a\") " pod="openshift-ingress/kube-auth-proxy-58ccc58984-75hc6"
Jul 30 03:49:40.424363 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.424337    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dd4c125-2623-4e9e-a868-f4ce0157d66a-tmp\") pod \"kube-auth-proxy-58ccc58984-75hc6\" (UID: \"8dd4c125-2623-4e9e-a868-f4ce0157d66a\") " pod="openshift-ingress/kube-auth-proxy-58ccc58984-75hc6"
Jul 30 03:49:40.425208 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.425183    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd4c125-2623-4e9e-a868-f4ce0157d66a-tls-certs\") pod \"kube-auth-proxy-58ccc58984-75hc6\" (UID: \"8dd4c125-2623-4e9e-a868-f4ce0157d66a\") " pod="openshift-ingress/kube-auth-proxy-58ccc58984-75hc6"
Jul 30 03:49:40.430404 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.430378    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j5hf\" (UniqueName: \"kubernetes.io/projected/8dd4c125-2623-4e9e-a868-f4ce0157d66a-kube-api-access-5j5hf\") pod \"kube-auth-proxy-58ccc58984-75hc6\" (UID: \"8dd4c125-2623-4e9e-a868-f4ce0157d66a\") " pod="openshift-ingress/kube-auth-proxy-58ccc58984-75hc6"
Jul 30 03:49:40.496993 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.496961    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:40.516124 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.516098    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58ccc58984-75hc6"
Jul 30 03:49:40.602487 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.602412    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:40.667762 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.667694    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58ccc58984-75hc6"]
Jul 30 03:49:40.670627 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:49:40.670595    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd4c125_2623_4e9e_a868_f4ce0157d66a.slice/crio-5b90dcd8b90bdaa7c75abe7492c3fd3eaa1fddb7ebad3d184c7a07494b3e1c33 WatchSource:0}: Error finding container 5b90dcd8b90bdaa7c75abe7492c3fd3eaa1fddb7ebad3d184c7a07494b3e1c33: Status 404 returned error can't find the container with id 5b90dcd8b90bdaa7c75abe7492c3fd3eaa1fddb7ebad3d184c7a07494b3e1c33
Jul 30 03:49:40.712860 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.712817    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58ccc58984-75hc6" event={"ID":"8dd4c125-2623-4e9e-a868-f4ce0157d66a","Type":"ContainerStarted","Data":"5b90dcd8b90bdaa7c75abe7492c3fd3eaa1fddb7ebad3d184c7a07494b3e1c33"}
Jul 30 03:49:40.893409 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:40.893370    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:41.046988 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:41.046958    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:41.428444 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:41.428406    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:41.493464 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:41.493420    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:42.420762 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:42.420724    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:43.731270 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:43.731191    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58ccc58984-75hc6" event={"ID":"8dd4c125-2623-4e9e-a868-f4ce0157d66a","Type":"ContainerStarted","Data":"5267bf0114bb2392224a0c4e2e49d8b6dde0aaa19a1ae1f84e07702409bf032a"}
Jul 30 03:49:43.746823 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:43.746776    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-58ccc58984-75hc6" podStartSLOduration=1.044566811 podStartE2EDuration="3.746761249s" totalImagesPullingTime="2.702194438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:49:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:49:43.744941706 +0000 UTC m=+282.783652725" watchObservedRunningTime="2026-07-30 03:49:43.746761249 +0000 UTC m=+282.785472268"
Jul 30 03:49:46.799799 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:46.799738    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:46.898917 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:46.898878    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:47.299916 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:47.299883    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:47.583836 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:47.583764    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:47.674069 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:47.674008    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:50.083003 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:50.082962    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:50.208499 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:50.208463    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:50.315387 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:50.315361    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:50.437703 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:50.437627    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:51.099639 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:51.099604    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:51.441772 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:51.441669    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:51.542770 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:51.542741    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:51.604162 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:49:51.604136    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:50:01.368244 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:01.368217    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 03:50:01.373800 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:01.373735    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 03:50:01.379067 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:01.379043    2569 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 03:50:13.785389 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:13.785351    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:50:41.110700 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:41.110663    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:50:41.208903 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:41.208868    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:50:41.267971 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:41.267944    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:50:56.588259 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.588187    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"]
Jul 30 03:50:56.591348 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.591332    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"
Jul 30 03:50:56.593434 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.593412    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 03:50:56.593434 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.593412    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 03:50:56.594055 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.594033    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7jnsd\""
Jul 30 03:50:56.598910 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.598890    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"]
Jul 30 03:50:56.694338 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.694314    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc5df\" (UniqueName: \"kubernetes.io/projected/9e1d39de-2961-4e11-b991-f2828bbae826-kube-api-access-hc5df\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg\" (UID: \"9e1d39de-2961-4e11-b991-f2828bbae826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"
Jul 30 03:50:56.694482 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.694351    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e1d39de-2961-4e11-b991-f2828bbae826-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg\" (UID: \"9e1d39de-2961-4e11-b991-f2828bbae826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"
Jul 30 03:50:56.694482 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.694407    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e1d39de-2961-4e11-b991-f2828bbae826-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg\" (UID: \"9e1d39de-2961-4e11-b991-f2828bbae826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"
Jul 30 03:50:56.795170 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.795141    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e1d39de-2961-4e11-b991-f2828bbae826-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg\" (UID: \"9e1d39de-2961-4e11-b991-f2828bbae826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"
Jul 30 03:50:56.795284 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.795186    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e1d39de-2961-4e11-b991-f2828bbae826-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg\" (UID: \"9e1d39de-2961-4e11-b991-f2828bbae826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"
Jul 30 03:50:56.795284 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.795223    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc5df\" (UniqueName: \"kubernetes.io/projected/9e1d39de-2961-4e11-b991-f2828bbae826-kube-api-access-hc5df\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg\" (UID: \"9e1d39de-2961-4e11-b991-f2828bbae826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"
Jul 30 03:50:56.795485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.795468    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e1d39de-2961-4e11-b991-f2828bbae826-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg\" (UID: \"9e1d39de-2961-4e11-b991-f2828bbae826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"
Jul 30 03:50:56.795566 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.795526    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e1d39de-2961-4e11-b991-f2828bbae826-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg\" (UID: \"9e1d39de-2961-4e11-b991-f2828bbae826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"
Jul 30 03:50:56.803218 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.803198    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc5df\" (UniqueName: \"kubernetes.io/projected/9e1d39de-2961-4e11-b991-f2828bbae826-kube-api-access-hc5df\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg\" (UID: \"9e1d39de-2961-4e11-b991-f2828bbae826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"
Jul 30 03:50:56.900939 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:56.900880    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"
Jul 30 03:50:57.025606 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.025576    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"]
Jul 30 03:50:57.028198 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:50:57.028174    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1d39de_2961_4e11_b991_f2828bbae826.slice/crio-e09d3030388fbd179588ca0d29a27c6e525de9b1478fa176b045d4bd975e65d0 WatchSource:0}: Error finding container e09d3030388fbd179588ca0d29a27c6e525de9b1478fa176b045d4bd975e65d0: Status 404 returned error can't find the container with id e09d3030388fbd179588ca0d29a27c6e525de9b1478fa176b045d4bd975e65d0
Jul 30 03:50:57.053860 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.053833    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg" event={"ID":"9e1d39de-2961-4e11-b991-f2828bbae826","Type":"ContainerStarted","Data":"e09d3030388fbd179588ca0d29a27c6e525de9b1478fa176b045d4bd975e65d0"}
Jul 30 03:50:57.187405 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.187349    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"]
Jul 30 03:50:57.194086 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.194065    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"
Jul 30 03:50:57.198295 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.198270    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"]
Jul 30 03:50:57.299811 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.299783    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40e22250-90e5-451e-b5ba-cba344f96343-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv\" (UID: \"40e22250-90e5-451e-b5ba-cba344f96343\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"
Jul 30 03:50:57.299936 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.299835    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n77np\" (UniqueName: \"kubernetes.io/projected/40e22250-90e5-451e-b5ba-cba344f96343-kube-api-access-n77np\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv\" (UID: \"40e22250-90e5-451e-b5ba-cba344f96343\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"
Jul 30 03:50:57.299936 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.299901    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40e22250-90e5-451e-b5ba-cba344f96343-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv\" (UID: \"40e22250-90e5-451e-b5ba-cba344f96343\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"
Jul 30 03:50:57.401039 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.401011    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n77np\" (UniqueName: \"kubernetes.io/projected/40e22250-90e5-451e-b5ba-cba344f96343-kube-api-access-n77np\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv\" (UID: \"40e22250-90e5-451e-b5ba-cba344f96343\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"
Jul 30 03:50:57.401151 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.401048    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40e22250-90e5-451e-b5ba-cba344f96343-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv\" (UID: \"40e22250-90e5-451e-b5ba-cba344f96343\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"
Jul 30 03:50:57.401151 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.401096    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40e22250-90e5-451e-b5ba-cba344f96343-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv\" (UID: \"40e22250-90e5-451e-b5ba-cba344f96343\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"
Jul 30 03:50:57.401576 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.401556    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40e22250-90e5-451e-b5ba-cba344f96343-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv\" (UID: \"40e22250-90e5-451e-b5ba-cba344f96343\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"
Jul 30 03:50:57.401576 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.401568    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40e22250-90e5-451e-b5ba-cba344f96343-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv\" (UID: \"40e22250-90e5-451e-b5ba-cba344f96343\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"
Jul 30 03:50:57.408394 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.408372    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n77np\" (UniqueName: \"kubernetes.io/projected/40e22250-90e5-451e-b5ba-cba344f96343-kube-api-access-n77np\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv\" (UID: \"40e22250-90e5-451e-b5ba-cba344f96343\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"
Jul 30 03:50:57.504192 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.504144    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"
Jul 30 03:50:57.586962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.586909    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"]
Jul 30 03:50:57.591180 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.591158    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"
Jul 30 03:50:57.597633 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.597606    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"]
Jul 30 03:50:57.632200 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.632179    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"]
Jul 30 03:50:57.634294 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:50:57.634264    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e22250_90e5_451e_b5ba_cba344f96343.slice/crio-ceea310d594c241d05a66cbc1cd8b22c13174d2acf44a2c3a6644e8c7922aa57 WatchSource:0}: Error finding container ceea310d594c241d05a66cbc1cd8b22c13174d2acf44a2c3a6644e8c7922aa57: Status 404 returned error can't find the container with id ceea310d594c241d05a66cbc1cd8b22c13174d2acf44a2c3a6644e8c7922aa57
Jul 30 03:50:57.703837 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.703810    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ae07334-cac6-46e3-a973-bc5eae480e07-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg\" (UID: \"5ae07334-cac6-46e3-a973-bc5eae480e07\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"
Jul 30 03:50:57.703946 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.703866    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b7hg\" (UniqueName: \"kubernetes.io/projected/5ae07334-cac6-46e3-a973-bc5eae480e07-kube-api-access-8b7hg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg\" (UID: \"5ae07334-cac6-46e3-a973-bc5eae480e07\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"
Jul 30 03:50:57.704014 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.703961    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ae07334-cac6-46e3-a973-bc5eae480e07-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg\" (UID: \"5ae07334-cac6-46e3-a973-bc5eae480e07\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"
Jul 30 03:50:57.805388 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.805358    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ae07334-cac6-46e3-a973-bc5eae480e07-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg\" (UID: \"5ae07334-cac6-46e3-a973-bc5eae480e07\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"
Jul 30 03:50:57.805496 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.805415    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b7hg\" (UniqueName: \"kubernetes.io/projected/5ae07334-cac6-46e3-a973-bc5eae480e07-kube-api-access-8b7hg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg\" (UID: \"5ae07334-cac6-46e3-a973-bc5eae480e07\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"
Jul 30 03:50:57.805496 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.805453    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ae07334-cac6-46e3-a973-bc5eae480e07-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg\" (UID: \"5ae07334-cac6-46e3-a973-bc5eae480e07\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"
Jul 30 03:50:57.805742 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.805719    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ae07334-cac6-46e3-a973-bc5eae480e07-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg\" (UID: \"5ae07334-cac6-46e3-a973-bc5eae480e07\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"
Jul 30 03:50:57.805855 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.805833    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ae07334-cac6-46e3-a973-bc5eae480e07-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg\" (UID: \"5ae07334-cac6-46e3-a973-bc5eae480e07\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"
Jul 30 03:50:57.813034 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.813013    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b7hg\" (UniqueName: \"kubernetes.io/projected/5ae07334-cac6-46e3-a973-bc5eae480e07-kube-api-access-8b7hg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg\" (UID: \"5ae07334-cac6-46e3-a973-bc5eae480e07\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"
Jul 30 03:50:57.903267 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.903243    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"
Jul 30 03:50:57.995936 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.995905    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"]
Jul 30 03:50:57.999559 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:57.999514    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"
Jul 30 03:50:58.006051 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.006030    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"]
Jul 30 03:50:58.032605 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.032584    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"]
Jul 30 03:50:58.034277 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:50:58.034255    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ae07334_cac6_46e3_a973_bc5eae480e07.slice/crio-45f1485d233e5664af2bf1be10f685f0c7080f29c88c95ac7d8ef67c0bea559c WatchSource:0}: Error finding container 45f1485d233e5664af2bf1be10f685f0c7080f29c88c95ac7d8ef67c0bea559c: Status 404 returned error can't find the container with id 45f1485d233e5664af2bf1be10f685f0c7080f29c88c95ac7d8ef67c0bea559c
Jul 30 03:50:58.058852 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.058800    2569 generic.go:356] "Generic (PLEG): container finished" podID="9e1d39de-2961-4e11-b991-f2828bbae826" containerID="f285fae97f7d4466e623197b6679c585b9f997910c167ae788d19bd16789d306" exitCode=0
Jul 30 03:50:58.058932 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.058843    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg" event={"ID":"9e1d39de-2961-4e11-b991-f2828bbae826","Type":"ContainerDied","Data":"f285fae97f7d4466e623197b6679c585b9f997910c167ae788d19bd16789d306"}
Jul 30 03:50:58.059325 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.059310    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 03:50:58.060407 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.060383    2569 generic.go:356] "Generic (PLEG): container finished" podID="40e22250-90e5-451e-b5ba-cba344f96343" containerID="2600f35961337aa8760498c14402a2374a1ba6ad7915595c14319d3424d538ee" exitCode=0
Jul 30 03:50:58.060486 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.060461    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv" event={"ID":"40e22250-90e5-451e-b5ba-cba344f96343","Type":"ContainerDied","Data":"2600f35961337aa8760498c14402a2374a1ba6ad7915595c14319d3424d538ee"}
Jul 30 03:50:58.060559 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.060501    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv" event={"ID":"40e22250-90e5-451e-b5ba-cba344f96343","Type":"ContainerStarted","Data":"ceea310d594c241d05a66cbc1cd8b22c13174d2acf44a2c3a6644e8c7922aa57"}
Jul 30 03:50:58.062053 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.062026    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg" event={"ID":"5ae07334-cac6-46e3-a973-bc5eae480e07","Type":"ContainerStarted","Data":"45f1485d233e5664af2bf1be10f685f0c7080f29c88c95ac7d8ef67c0bea559c"}
Jul 30 03:50:58.106888 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.106862    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a1152d-bee4-464f-b723-c71b08f6753c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr\" (UID: \"08a1152d-bee4-464f-b723-c71b08f6753c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"
Jul 30 03:50:58.107001 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.106906    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a1152d-bee4-464f-b723-c71b08f6753c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr\" (UID: \"08a1152d-bee4-464f-b723-c71b08f6753c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"
Jul 30 03:50:58.107062 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.107009    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr8cb\" (UniqueName: \"kubernetes.io/projected/08a1152d-bee4-464f-b723-c71b08f6753c-kube-api-access-kr8cb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr\" (UID: \"08a1152d-bee4-464f-b723-c71b08f6753c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"
Jul 30 03:50:58.208201 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.208173    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr8cb\" (UniqueName: \"kubernetes.io/projected/08a1152d-bee4-464f-b723-c71b08f6753c-kube-api-access-kr8cb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr\" (UID: \"08a1152d-bee4-464f-b723-c71b08f6753c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"
Jul 30 03:50:58.208295 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.208236    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a1152d-bee4-464f-b723-c71b08f6753c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr\" (UID: \"08a1152d-bee4-464f-b723-c71b08f6753c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"
Jul 30 03:50:58.208295 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.208269    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a1152d-bee4-464f-b723-c71b08f6753c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr\" (UID: \"08a1152d-bee4-464f-b723-c71b08f6753c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"
Jul 30 03:50:58.208649 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.208630    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a1152d-bee4-464f-b723-c71b08f6753c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr\" (UID: \"08a1152d-bee4-464f-b723-c71b08f6753c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"
Jul 30 03:50:58.208686 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.208651    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a1152d-bee4-464f-b723-c71b08f6753c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr\" (UID: \"08a1152d-bee4-464f-b723-c71b08f6753c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"
Jul 30 03:50:58.215780 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.215761    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr8cb\" (UniqueName: \"kubernetes.io/projected/08a1152d-bee4-464f-b723-c71b08f6753c-kube-api-access-kr8cb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr\" (UID: \"08a1152d-bee4-464f-b723-c71b08f6753c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"
Jul 30 03:50:58.312060 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.312008    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"
Jul 30 03:50:58.437784 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:58.437762    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"]
Jul 30 03:50:58.439834 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:50:58.439808    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a1152d_bee4_464f_b723_c71b08f6753c.slice/crio-3f966a94ffbcc469c9ed0d8df1780a073aa9c0b7c3ab17a4688b52053cf77376 WatchSource:0}: Error finding container 3f966a94ffbcc469c9ed0d8df1780a073aa9c0b7c3ab17a4688b52053cf77376: Status 404 returned error can't find the container with id 3f966a94ffbcc469c9ed0d8df1780a073aa9c0b7c3ab17a4688b52053cf77376
Jul 30 03:50:59.069646 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:59.069617    2569 generic.go:356] "Generic (PLEG): container finished" podID="08a1152d-bee4-464f-b723-c71b08f6753c" containerID="62c7aed8156b48faa8db25ac1f19a1fa9870d580df47b33e44f91bf754425701" exitCode=0
Jul 30 03:50:59.069965 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:59.069722    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr" event={"ID":"08a1152d-bee4-464f-b723-c71b08f6753c","Type":"ContainerDied","Data":"62c7aed8156b48faa8db25ac1f19a1fa9870d580df47b33e44f91bf754425701"}
Jul 30 03:50:59.069965 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:59.069765    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr" event={"ID":"08a1152d-bee4-464f-b723-c71b08f6753c","Type":"ContainerStarted","Data":"3f966a94ffbcc469c9ed0d8df1780a073aa9c0b7c3ab17a4688b52053cf77376"}
Jul 30 03:50:59.072018 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:59.071993    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv" event={"ID":"40e22250-90e5-451e-b5ba-cba344f96343","Type":"ContainerStarted","Data":"81497d62052fdc257e00e517aa7e2bae66234c7aac8844c0bf6a5b1ff4c61687"}
Jul 30 03:50:59.074150 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:59.074128    2569 generic.go:356] "Generic (PLEG): container finished" podID="5ae07334-cac6-46e3-a973-bc5eae480e07" containerID="0795ce820b832d301d3805c4071e18e013a4e8603feb00bf365d48bd65c63a59" exitCode=0
Jul 30 03:50:59.074281 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:50:59.074161    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg" event={"ID":"5ae07334-cac6-46e3-a973-bc5eae480e07","Type":"ContainerDied","Data":"0795ce820b832d301d3805c4071e18e013a4e8603feb00bf365d48bd65c63a59"}
Jul 30 03:51:00.081365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:00.081329    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr" event={"ID":"08a1152d-bee4-464f-b723-c71b08f6753c","Type":"ContainerStarted","Data":"b467a0d0cbe3c7a0018d6b7ca0f7800e802aa3fd0006b66cfbdea4e0bc0fc3a3"}
Jul 30 03:51:00.083131 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:00.083106    2569 generic.go:356] "Generic (PLEG): container finished" podID="40e22250-90e5-451e-b5ba-cba344f96343" containerID="81497d62052fdc257e00e517aa7e2bae66234c7aac8844c0bf6a5b1ff4c61687" exitCode=0
Jul 30 03:51:00.083216 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:00.083188    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv" event={"ID":"40e22250-90e5-451e-b5ba-cba344f96343","Type":"ContainerDied","Data":"81497d62052fdc257e00e517aa7e2bae66234c7aac8844c0bf6a5b1ff4c61687"}
Jul 30 03:51:00.085111 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:00.085090    2569 generic.go:356] "Generic (PLEG): container finished" podID="5ae07334-cac6-46e3-a973-bc5eae480e07" containerID="ea5b9cfa33ac1aaea78cc7434aec425240ac1622ce17a57399f1e8f0ed654457" exitCode=0
Jul 30 03:51:00.085195 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:00.085172    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg" event={"ID":"5ae07334-cac6-46e3-a973-bc5eae480e07","Type":"ContainerDied","Data":"ea5b9cfa33ac1aaea78cc7434aec425240ac1622ce17a57399f1e8f0ed654457"}
Jul 30 03:51:00.086962 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:00.086944    2569 generic.go:356] "Generic (PLEG): container finished" podID="9e1d39de-2961-4e11-b991-f2828bbae826" containerID="8e86cc9aae82a22404c4350e1b64a5bb9327092d02997c0e001d81f502bba4a1" exitCode=0
Jul 30 03:51:00.087070 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:00.086975    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg" event={"ID":"9e1d39de-2961-4e11-b991-f2828bbae826","Type":"ContainerDied","Data":"8e86cc9aae82a22404c4350e1b64a5bb9327092d02997c0e001d81f502bba4a1"}
Jul 30 03:51:01.094094 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:01.094059    2569 generic.go:356] "Generic (PLEG): container finished" podID="40e22250-90e5-451e-b5ba-cba344f96343" containerID="57d3a91db8f286193e53f86e14fcd8ce4f987a3546cb0dc978734e59470c2458" exitCode=0
Jul 30 03:51:01.094523 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:01.094133    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv" event={"ID":"40e22250-90e5-451e-b5ba-cba344f96343","Type":"ContainerDied","Data":"57d3a91db8f286193e53f86e14fcd8ce4f987a3546cb0dc978734e59470c2458"}
Jul 30 03:51:01.096158 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:01.096132    2569 generic.go:356] "Generic (PLEG): container finished" podID="5ae07334-cac6-46e3-a973-bc5eae480e07" containerID="ce21570b25a979380b88e08fcf498fedb994d2586887b7342040784b9771b228" exitCode=0
Jul 30 03:51:01.096272 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:01.096214    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg" event={"ID":"5ae07334-cac6-46e3-a973-bc5eae480e07","Type":"ContainerDied","Data":"ce21570b25a979380b88e08fcf498fedb994d2586887b7342040784b9771b228"}
Jul 30 03:51:01.098309 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:01.098287    2569 generic.go:356] "Generic (PLEG): container finished" podID="9e1d39de-2961-4e11-b991-f2828bbae826" containerID="fd6e625de9b52f6de1d4b74f4961bcd78515536764138911b2fe81c3cff20775" exitCode=0
Jul 30 03:51:01.098400 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:01.098327    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg" event={"ID":"9e1d39de-2961-4e11-b991-f2828bbae826","Type":"ContainerDied","Data":"fd6e625de9b52f6de1d4b74f4961bcd78515536764138911b2fe81c3cff20775"}
Jul 30 03:51:01.099938 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:01.099920    2569 generic.go:356] "Generic (PLEG): container finished" podID="08a1152d-bee4-464f-b723-c71b08f6753c" containerID="b467a0d0cbe3c7a0018d6b7ca0f7800e802aa3fd0006b66cfbdea4e0bc0fc3a3" exitCode=0
Jul 30 03:51:01.100010 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:01.099952    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr" event={"ID":"08a1152d-bee4-464f-b723-c71b08f6753c","Type":"ContainerDied","Data":"b467a0d0cbe3c7a0018d6b7ca0f7800e802aa3fd0006b66cfbdea4e0bc0fc3a3"}
Jul 30 03:51:02.106656 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.106615    2569 generic.go:356] "Generic (PLEG): container finished" podID="08a1152d-bee4-464f-b723-c71b08f6753c" containerID="e905f07601c6c742692763b356920adf01034077aabfe9750da1c8a0ee560542" exitCode=0
Jul 30 03:51:02.107124 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.106700    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr" event={"ID":"08a1152d-bee4-464f-b723-c71b08f6753c","Type":"ContainerDied","Data":"e905f07601c6c742692763b356920adf01034077aabfe9750da1c8a0ee560542"}
Jul 30 03:51:02.257752 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.257719    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"
Jul 30 03:51:02.298917 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.298896    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"
Jul 30 03:51:02.301212 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.301194    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"
Jul 30 03:51:02.342979 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.342953    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e1d39de-2961-4e11-b991-f2828bbae826-util\") pod \"9e1d39de-2961-4e11-b991-f2828bbae826\" (UID: \"9e1d39de-2961-4e11-b991-f2828bbae826\") "
Jul 30 03:51:02.343131 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.343020    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc5df\" (UniqueName: \"kubernetes.io/projected/9e1d39de-2961-4e11-b991-f2828bbae826-kube-api-access-hc5df\") pod \"9e1d39de-2961-4e11-b991-f2828bbae826\" (UID: \"9e1d39de-2961-4e11-b991-f2828bbae826\") "
Jul 30 03:51:02.343131 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.343061    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e1d39de-2961-4e11-b991-f2828bbae826-bundle\") pod \"9e1d39de-2961-4e11-b991-f2828bbae826\" (UID: \"9e1d39de-2961-4e11-b991-f2828bbae826\") "
Jul 30 03:51:02.343562 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.343519    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e1d39de-2961-4e11-b991-f2828bbae826-bundle" (OuterVolumeSpecName: "bundle") pod "9e1d39de-2961-4e11-b991-f2828bbae826" (UID: "9e1d39de-2961-4e11-b991-f2828bbae826"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:51:02.345322 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.345299    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e1d39de-2961-4e11-b991-f2828bbae826-kube-api-access-hc5df" (OuterVolumeSpecName: "kube-api-access-hc5df") pod "9e1d39de-2961-4e11-b991-f2828bbae826" (UID: "9e1d39de-2961-4e11-b991-f2828bbae826"). InnerVolumeSpecName "kube-api-access-hc5df". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:51:02.401210 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.401160    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e1d39de-2961-4e11-b991-f2828bbae826-util" (OuterVolumeSpecName: "util") pod "9e1d39de-2961-4e11-b991-f2828bbae826" (UID: "9e1d39de-2961-4e11-b991-f2828bbae826"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:51:02.444320 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.444299    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8b7hg\" (UniqueName: \"kubernetes.io/projected/5ae07334-cac6-46e3-a973-bc5eae480e07-kube-api-access-8b7hg\") pod \"5ae07334-cac6-46e3-a973-bc5eae480e07\" (UID: \"5ae07334-cac6-46e3-a973-bc5eae480e07\") "
Jul 30 03:51:02.444388 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.444329    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n77np\" (UniqueName: \"kubernetes.io/projected/40e22250-90e5-451e-b5ba-cba344f96343-kube-api-access-n77np\") pod \"40e22250-90e5-451e-b5ba-cba344f96343\" (UID: \"40e22250-90e5-451e-b5ba-cba344f96343\") "
Jul 30 03:51:02.444388 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.444359    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ae07334-cac6-46e3-a973-bc5eae480e07-bundle\") pod \"5ae07334-cac6-46e3-a973-bc5eae480e07\" (UID: \"5ae07334-cac6-46e3-a973-bc5eae480e07\") "
Jul 30 03:51:02.444388 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.444382    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40e22250-90e5-451e-b5ba-cba344f96343-util\") pod \"40e22250-90e5-451e-b5ba-cba344f96343\" (UID: \"40e22250-90e5-451e-b5ba-cba344f96343\") "
Jul 30 03:51:02.444573 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.444432    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40e22250-90e5-451e-b5ba-cba344f96343-bundle\") pod \"40e22250-90e5-451e-b5ba-cba344f96343\" (UID: \"40e22250-90e5-451e-b5ba-cba344f96343\") "
Jul 30 03:51:02.444573 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.444472    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ae07334-cac6-46e3-a973-bc5eae480e07-util\") pod \"5ae07334-cac6-46e3-a973-bc5eae480e07\" (UID: \"5ae07334-cac6-46e3-a973-bc5eae480e07\") "
Jul 30 03:51:02.444719 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.444678    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e1d39de-2961-4e11-b991-f2828bbae826-util\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:51:02.444719 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.444699    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc5df\" (UniqueName: \"kubernetes.io/projected/9e1d39de-2961-4e11-b991-f2828bbae826-kube-api-access-hc5df\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:51:02.444719 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.444713    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e1d39de-2961-4e11-b991-f2828bbae826-bundle\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:51:02.444999 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.444972    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ae07334-cac6-46e3-a973-bc5eae480e07-bundle" (OuterVolumeSpecName: "bundle") pod "5ae07334-cac6-46e3-a973-bc5eae480e07" (UID: "5ae07334-cac6-46e3-a973-bc5eae480e07"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:51:02.445158 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.445134    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e22250-90e5-451e-b5ba-cba344f96343-bundle" (OuterVolumeSpecName: "bundle") pod "40e22250-90e5-451e-b5ba-cba344f96343" (UID: "40e22250-90e5-451e-b5ba-cba344f96343"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:51:02.447021 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.447000    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40e22250-90e5-451e-b5ba-cba344f96343-kube-api-access-n77np" (OuterVolumeSpecName: "kube-api-access-n77np") pod "40e22250-90e5-451e-b5ba-cba344f96343" (UID: "40e22250-90e5-451e-b5ba-cba344f96343"). InnerVolumeSpecName "kube-api-access-n77np". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:51:02.447109 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.447030    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ae07334-cac6-46e3-a973-bc5eae480e07-kube-api-access-8b7hg" (OuterVolumeSpecName: "kube-api-access-8b7hg") pod "5ae07334-cac6-46e3-a973-bc5eae480e07" (UID: "5ae07334-cac6-46e3-a973-bc5eae480e07"). InnerVolumeSpecName "kube-api-access-8b7hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:51:02.453469 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.453447    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e22250-90e5-451e-b5ba-cba344f96343-util" (OuterVolumeSpecName: "util") pod "40e22250-90e5-451e-b5ba-cba344f96343" (UID: "40e22250-90e5-451e-b5ba-cba344f96343"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:51:02.453617 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.453595    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ae07334-cac6-46e3-a973-bc5eae480e07-util" (OuterVolumeSpecName: "util") pod "5ae07334-cac6-46e3-a973-bc5eae480e07" (UID: "5ae07334-cac6-46e3-a973-bc5eae480e07"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:51:02.545977 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.545881    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40e22250-90e5-451e-b5ba-cba344f96343-bundle\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:51:02.545977 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.545908    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ae07334-cac6-46e3-a973-bc5eae480e07-util\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:51:02.545977 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.545924    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8b7hg\" (UniqueName: \"kubernetes.io/projected/5ae07334-cac6-46e3-a973-bc5eae480e07-kube-api-access-8b7hg\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:51:02.545977 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.545940    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n77np\" (UniqueName: \"kubernetes.io/projected/40e22250-90e5-451e-b5ba-cba344f96343-kube-api-access-n77np\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:51:02.545977 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.545954    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ae07334-cac6-46e3-a973-bc5eae480e07-bundle\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:51:02.545977 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:02.545967    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40e22250-90e5-451e-b5ba-cba344f96343-util\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:51:03.113415 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.113385    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg"
Jul 30 03:51:03.113831 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.113386    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg" event={"ID":"9e1d39de-2961-4e11-b991-f2828bbae826","Type":"ContainerDied","Data":"e09d3030388fbd179588ca0d29a27c6e525de9b1478fa176b045d4bd975e65d0"}
Jul 30 03:51:03.113831 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.113513    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e09d3030388fbd179588ca0d29a27c6e525de9b1478fa176b045d4bd975e65d0"
Jul 30 03:51:03.115461 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.115436    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv" event={"ID":"40e22250-90e5-451e-b5ba-cba344f96343","Type":"ContainerDied","Data":"ceea310d594c241d05a66cbc1cd8b22c13174d2acf44a2c3a6644e8c7922aa57"}
Jul 30 03:51:03.115461 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.115454    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv"
Jul 30 03:51:03.115657 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.115474    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ceea310d594c241d05a66cbc1cd8b22c13174d2acf44a2c3a6644e8c7922aa57"
Jul 30 03:51:03.117308 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.117280    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg" event={"ID":"5ae07334-cac6-46e3-a973-bc5eae480e07","Type":"ContainerDied","Data":"45f1485d233e5664af2bf1be10f685f0c7080f29c88c95ac7d8ef67c0bea559c"}
Jul 30 03:51:03.117400 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.117324    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45f1485d233e5664af2bf1be10f685f0c7080f29c88c95ac7d8ef67c0bea559c"
Jul 30 03:51:03.117474 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.117459    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg"
Jul 30 03:51:03.234708 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.234685    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"
Jul 30 03:51:03.352422 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.352396    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a1152d-bee4-464f-b723-c71b08f6753c-util\") pod \"08a1152d-bee4-464f-b723-c71b08f6753c\" (UID: \"08a1152d-bee4-464f-b723-c71b08f6753c\") "
Jul 30 03:51:03.352422 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.352426    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kr8cb\" (UniqueName: \"kubernetes.io/projected/08a1152d-bee4-464f-b723-c71b08f6753c-kube-api-access-kr8cb\") pod \"08a1152d-bee4-464f-b723-c71b08f6753c\" (UID: \"08a1152d-bee4-464f-b723-c71b08f6753c\") "
Jul 30 03:51:03.352640 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.352477    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a1152d-bee4-464f-b723-c71b08f6753c-bundle\") pod \"08a1152d-bee4-464f-b723-c71b08f6753c\" (UID: \"08a1152d-bee4-464f-b723-c71b08f6753c\") "
Jul 30 03:51:03.353000 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.352975    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08a1152d-bee4-464f-b723-c71b08f6753c-bundle" (OuterVolumeSpecName: "bundle") pod "08a1152d-bee4-464f-b723-c71b08f6753c" (UID: "08a1152d-bee4-464f-b723-c71b08f6753c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:51:03.354721 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.354699    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08a1152d-bee4-464f-b723-c71b08f6753c-kube-api-access-kr8cb" (OuterVolumeSpecName: "kube-api-access-kr8cb") pod "08a1152d-bee4-464f-b723-c71b08f6753c" (UID: "08a1152d-bee4-464f-b723-c71b08f6753c"). InnerVolumeSpecName "kube-api-access-kr8cb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:51:03.358067 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.358042    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08a1152d-bee4-464f-b723-c71b08f6753c-util" (OuterVolumeSpecName: "util") pod "08a1152d-bee4-464f-b723-c71b08f6753c" (UID: "08a1152d-bee4-464f-b723-c71b08f6753c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:51:03.453246 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.453172    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a1152d-bee4-464f-b723-c71b08f6753c-bundle\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:51:03.453246 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.453204    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a1152d-bee4-464f-b723-c71b08f6753c-util\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:51:03.453246 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:03.453218    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kr8cb\" (UniqueName: \"kubernetes.io/projected/08a1152d-bee4-464f-b723-c71b08f6753c-kube-api-access-kr8cb\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:51:04.124123 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:04.124099    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr"
Jul 30 03:51:04.124518 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:04.124128    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr" event={"ID":"08a1152d-bee4-464f-b723-c71b08f6753c","Type":"ContainerDied","Data":"3f966a94ffbcc469c9ed0d8df1780a073aa9c0b7c3ab17a4688b52053cf77376"}
Jul 30 03:51:04.124518 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:04.124181    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f966a94ffbcc469c9ed0d8df1780a073aa9c0b7c3ab17a4688b52053cf77376"
Jul 30 03:51:09.892332 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:09.892290    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:14.100013 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:14.099951    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:16.479034 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:16.478980    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:16.719394 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:16.719357    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:17.080440 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:17.080408    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:20.512292 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:20.512259    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:21.551963 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:21.551899    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:21.663414 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:21.663386    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:23.587874 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:23.587823    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:25.456100 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:25.456060    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:26.699945 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:26.699911    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:26.801472 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:26.801440    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:30.491949 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:30.491908    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:30.592436 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:30.592400    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:32.348778 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:32.348743    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:32.672783 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:32.672696    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:34.686280 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:34.686246    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:34.793989 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:34.793954    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:35.101436 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:35.101399    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:35.201576 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:35.201550    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:36.270410 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:36.270378    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:38.047485 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:38.047449    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:38.147772 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:38.147747    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:42.437375 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437340    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"]
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437669    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e1d39de-2961-4e11-b991-f2828bbae826" containerName="extract"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437681    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e1d39de-2961-4e11-b991-f2828bbae826" containerName="extract"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437692    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08a1152d-bee4-464f-b723-c71b08f6753c" containerName="extract"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437698    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="08a1152d-bee4-464f-b723-c71b08f6753c" containerName="extract"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437710    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae07334-cac6-46e3-a973-bc5eae480e07" containerName="util"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437716    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae07334-cac6-46e3-a973-bc5eae480e07" containerName="util"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437724    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40e22250-90e5-451e-b5ba-cba344f96343" containerName="extract"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437729    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="40e22250-90e5-451e-b5ba-cba344f96343" containerName="extract"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437737    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40e22250-90e5-451e-b5ba-cba344f96343" containerName="util"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437742    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="40e22250-90e5-451e-b5ba-cba344f96343" containerName="util"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437749    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e1d39de-2961-4e11-b991-f2828bbae826" containerName="pull"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437754    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e1d39de-2961-4e11-b991-f2828bbae826" containerName="pull"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437761    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08a1152d-bee4-464f-b723-c71b08f6753c" containerName="util"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437766    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="08a1152d-bee4-464f-b723-c71b08f6753c" containerName="util"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437771    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40e22250-90e5-451e-b5ba-cba344f96343" containerName="pull"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437776    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="40e22250-90e5-451e-b5ba-cba344f96343" containerName="pull"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437781    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08a1152d-bee4-464f-b723-c71b08f6753c" containerName="pull"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437799    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="08a1152d-bee4-464f-b723-c71b08f6753c" containerName="pull"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437809    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e1d39de-2961-4e11-b991-f2828bbae826" containerName="util"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437814    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e1d39de-2961-4e11-b991-f2828bbae826" containerName="util"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437820    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae07334-cac6-46e3-a973-bc5eae480e07" containerName="pull"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437825    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae07334-cac6-46e3-a973-bc5eae480e07" containerName="pull"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437832    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae07334-cac6-46e3-a973-bc5eae480e07" containerName="extract"
Jul 30 03:51:42.437847 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437836    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae07334-cac6-46e3-a973-bc5eae480e07" containerName="extract"
Jul 30 03:51:42.438660 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437882    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ae07334-cac6-46e3-a973-bc5eae480e07" containerName="extract"
Jul 30 03:51:42.438660 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437890    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="08a1152d-bee4-464f-b723-c71b08f6753c" containerName="extract"
Jul 30 03:51:42.438660 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437897    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e1d39de-2961-4e11-b991-f2828bbae826" containerName="extract"
Jul 30 03:51:42.438660 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.437903    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="40e22250-90e5-451e-b5ba-cba344f96343" containerName="extract"
Jul 30 03:51:42.441028 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.441011    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"
Jul 30 03:51:42.443245 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.443226    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 03:51:42.443372 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.443302    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zhdjg\""
Jul 30 03:51:42.443470 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.443384    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 03:51:42.453585 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.453563    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"]
Jul 30 03:51:42.559092 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.559070    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a3adf3c-282f-4c72-97a0-343b8b132f5b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sghmh\" (UID: \"0a3adf3c-282f-4c72-97a0-343b8b132f5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"
Jul 30 03:51:42.559200 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.559125    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75g78\" (UniqueName: \"kubernetes.io/projected/0a3adf3c-282f-4c72-97a0-343b8b132f5b-kube-api-access-75g78\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sghmh\" (UID: \"0a3adf3c-282f-4c72-97a0-343b8b132f5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"
Jul 30 03:51:42.659616 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.659595    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75g78\" (UniqueName: \"kubernetes.io/projected/0a3adf3c-282f-4c72-97a0-343b8b132f5b-kube-api-access-75g78\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sghmh\" (UID: \"0a3adf3c-282f-4c72-97a0-343b8b132f5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"
Jul 30 03:51:42.659728 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.659641    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a3adf3c-282f-4c72-97a0-343b8b132f5b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sghmh\" (UID: \"0a3adf3c-282f-4c72-97a0-343b8b132f5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"
Jul 30 03:51:42.660006 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.659988    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a3adf3c-282f-4c72-97a0-343b8b132f5b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sghmh\" (UID: \"0a3adf3c-282f-4c72-97a0-343b8b132f5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"
Jul 30 03:51:42.677526 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.677501    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75g78\" (UniqueName: \"kubernetes.io/projected/0a3adf3c-282f-4c72-97a0-343b8b132f5b-kube-api-access-75g78\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sghmh\" (UID: \"0a3adf3c-282f-4c72-97a0-343b8b132f5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"
Jul 30 03:51:42.751121 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.751070    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"
Jul 30 03:51:42.882764 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:42.882736    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"]
Jul 30 03:51:42.884656 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:51:42.884631    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a3adf3c_282f_4c72_97a0_343b8b132f5b.slice/crio-3aa4531f1acd842c4c138d145a03f0e6e15385a7aacc5f9dc2c2659a9e91c8f7 WatchSource:0}: Error finding container 3aa4531f1acd842c4c138d145a03f0e6e15385a7aacc5f9dc2c2659a9e91c8f7: Status 404 returned error can't find the container with id 3aa4531f1acd842c4c138d145a03f0e6e15385a7aacc5f9dc2c2659a9e91c8f7
Jul 30 03:51:43.132837 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:43.132805    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"]
Jul 30 03:51:43.142916 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:43.142876    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"]
Jul 30 03:51:43.162682 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:43.162657    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"]
Jul 30 03:51:43.167037 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:43.167020    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"
Jul 30 03:51:43.192976 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:43.192954    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"]
Jul 30 03:51:43.266004 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:43.265974    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a509a2d-bed0-452c-baf8-ea326bd2c52c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-l2hpp\" (UID: \"3a509a2d-bed0-452c-baf8-ea326bd2c52c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"
Jul 30 03:51:43.266171 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:43.266041    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh4qt\" (UniqueName: \"kubernetes.io/projected/3a509a2d-bed0-452c-baf8-ea326bd2c52c-kube-api-access-bh4qt\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-l2hpp\" (UID: \"3a509a2d-bed0-452c-baf8-ea326bd2c52c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"
Jul 30 03:51:43.367119 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:43.367084    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh4qt\" (UniqueName: \"kubernetes.io/projected/3a509a2d-bed0-452c-baf8-ea326bd2c52c-kube-api-access-bh4qt\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-l2hpp\" (UID: \"3a509a2d-bed0-452c-baf8-ea326bd2c52c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"
Jul 30 03:51:43.367314 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:43.367143    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a509a2d-bed0-452c-baf8-ea326bd2c52c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-l2hpp\" (UID: \"3a509a2d-bed0-452c-baf8-ea326bd2c52c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"
Jul 30 03:51:43.367497 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:43.367479    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a509a2d-bed0-452c-baf8-ea326bd2c52c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-l2hpp\" (UID: \"3a509a2d-bed0-452c-baf8-ea326bd2c52c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"
Jul 30 03:51:43.379192 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:43.379172    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh4qt\" (UniqueName: \"kubernetes.io/projected/3a509a2d-bed0-452c-baf8-ea326bd2c52c-kube-api-access-bh4qt\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-l2hpp\" (UID: \"3a509a2d-bed0-452c-baf8-ea326bd2c52c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"
Jul 30 03:51:43.478715 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:43.478652    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"
Jul 30 03:51:43.617014 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:43.616990    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"]
Jul 30 03:51:43.624364 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:51:43.624333    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a509a2d_bed0_452c_baf8_ea326bd2c52c.slice/crio-70aa72c0ff5641f0eb2752da7024254e2ce2dbbeade5650fc403a362158c2dd3 WatchSource:0}: Error finding container 70aa72c0ff5641f0eb2752da7024254e2ce2dbbeade5650fc403a362158c2dd3: Status 404 returned error can't find the container with id 70aa72c0ff5641f0eb2752da7024254e2ce2dbbeade5650fc403a362158c2dd3
Jul 30 03:51:44.311456 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:44.311378    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp" event={"ID":"3a509a2d-bed0-452c-baf8-ea326bd2c52c","Type":"ContainerStarted","Data":"70aa72c0ff5641f0eb2752da7024254e2ce2dbbeade5650fc403a362158c2dd3"}
Jul 30 03:51:44.633172 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:44.633088    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:44.732998 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:44.732942    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:47.441342 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.441301    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:47.541605 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.541573    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:47.730099 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.730023    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6868887c67-dklsc"]
Jul 30 03:51:47.730347 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.730334    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a3adf3c-282f-4c72-97a0-343b8b132f5b" containerName="manager"
Jul 30 03:51:47.730386 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.730349    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a3adf3c-282f-4c72-97a0-343b8b132f5b" containerName="manager"
Jul 30 03:51:47.730421 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.730417    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a3adf3c-282f-4c72-97a0-343b8b132f5b" containerName="manager"
Jul 30 03:51:47.733486 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.733464    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.742863 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.742774    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6868887c67-dklsc"]
Jul 30 03:51:47.786621 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.786588    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:47.804709 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.804676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-oauth-serving-cert\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.804845 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.804739    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-service-ca\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.804845 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.804801    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-console-serving-cert\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.804845 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.804836    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-console-oauth-config\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.805002 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.804890    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-console-config\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.805002 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.804938    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v42q7\" (UniqueName: \"kubernetes.io/projected/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-kube-api-access-v42q7\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.805002 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.804988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-trusted-ca-bundle\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.906120 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.906095    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-console-serving-cert\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.906245 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.906123    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-console-oauth-config\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.906245 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.906152    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-console-config\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.906245 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.906172    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v42q7\" (UniqueName: \"kubernetes.io/projected/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-kube-api-access-v42q7\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.906245 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.906190    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-trusted-ca-bundle\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.906245 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.906223    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-oauth-serving-cert\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.906514 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.906273    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-service-ca\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.906952 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.906931    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-console-config\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.907050 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.906988    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-service-ca\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.907098 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.907051    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-trusted-ca-bundle\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.907261 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.907236    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-oauth-serving-cert\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.908703 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.908677    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-console-oauth-config\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.908807 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.908782    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-console-serving-cert\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:47.914275 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:47.914255    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v42q7\" (UniqueName: \"kubernetes.io/projected/ade1b5c1-d49f-40f2-8ac6-54419a36d58b-kube-api-access-v42q7\") pod \"console-6868887c67-dklsc\" (UID: \"ade1b5c1-d49f-40f2-8ac6-54419a36d58b\") " pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:48.046563 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.046515    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:48.179296 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.179273    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6868887c67-dklsc"]
Jul 30 03:51:48.181281 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:51:48.181251    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podade1b5c1_d49f_40f2_8ac6_54419a36d58b.slice/crio-355c700c17122d68fe800e72b9aee7bb73a795d501ef3974f13e44a76f9ea4a1 WatchSource:0}: Error finding container 355c700c17122d68fe800e72b9aee7bb73a795d501ef3974f13e44a76f9ea4a1: Status 404 returned error can't find the container with id 355c700c17122d68fe800e72b9aee7bb73a795d501ef3974f13e44a76f9ea4a1
Jul 30 03:51:48.336342 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.336221    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh" podUID="0a3adf3c-282f-4c72-97a0-343b8b132f5b" containerName="manager" containerID="cri-o://2dace548b3b4fdcf6519a4b7e4ec75e754fd0332f18e70c23a0a19f546d8ffb3" gracePeriod=2
Jul 30 03:51:48.337814 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.337772    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6868887c67-dklsc" event={"ID":"ade1b5c1-d49f-40f2-8ac6-54419a36d58b","Type":"ContainerStarted","Data":"c564d53d1e1940f74ef6f4ee010980ab751ce897c91f2d71c7609a663a181fe9"}
Jul 30 03:51:48.337940 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.337826    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6868887c67-dklsc" event={"ID":"ade1b5c1-d49f-40f2-8ac6-54419a36d58b","Type":"ContainerStarted","Data":"355c700c17122d68fe800e72b9aee7bb73a795d501ef3974f13e44a76f9ea4a1"}
Jul 30 03:51:48.339570 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.339516    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp" event={"ID":"3a509a2d-bed0-452c-baf8-ea326bd2c52c","Type":"ContainerStarted","Data":"77100b65ce500ae6c6dab8c2fa7e6a2feec6b6f3b005dd55779459f48d987a55"}
Jul 30 03:51:48.339683 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.339633    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"
Jul 30 03:51:48.356726 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.356683    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6868887c67-dklsc" podStartSLOduration=1.356668766 podStartE2EDuration="1.356668766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:51:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:51:48.353942872 +0000 UTC m=+407.392653902" watchObservedRunningTime="2026-07-30 03:51:48.356668766 +0000 UTC m=+407.395379785"
Jul 30 03:51:48.371052 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.371002    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp" podStartSLOduration=1.459429112 podStartE2EDuration="5.370989171s" totalImagesPullingTime="3.911560059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:51:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:51:48.369339681 +0000 UTC m=+407.408050692" watchObservedRunningTime="2026-07-30 03:51:48.370989171 +0000 UTC m=+407.409700191"
Jul 30 03:51:48.574261 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.574236    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"
Jul 30 03:51:48.576209 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:51:48.576182    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sghmh\" is forbidden: User \"system:node:ip-10-0-136-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-104.ec2.internal' and this object" podUID="0a3adf3c-282f-4c72-97a0-343b8b132f5b" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"
Jul 30 03:51:48.712804 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.712729    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75g78\" (UniqueName: \"kubernetes.io/projected/0a3adf3c-282f-4c72-97a0-343b8b132f5b-kube-api-access-75g78\") pod \"0a3adf3c-282f-4c72-97a0-343b8b132f5b\" (UID: \"0a3adf3c-282f-4c72-97a0-343b8b132f5b\") "
Jul 30 03:51:48.712939 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.712820    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a3adf3c-282f-4c72-97a0-343b8b132f5b-extensions-socket-volume\") pod \"0a3adf3c-282f-4c72-97a0-343b8b132f5b\" (UID: \"0a3adf3c-282f-4c72-97a0-343b8b132f5b\") "
Jul 30 03:51:48.713093 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.713067    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a3adf3c-282f-4c72-97a0-343b8b132f5b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0a3adf3c-282f-4c72-97a0-343b8b132f5b" (UID: "0a3adf3c-282f-4c72-97a0-343b8b132f5b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:51:48.714797 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.714779    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a3adf3c-282f-4c72-97a0-343b8b132f5b-kube-api-access-75g78" (OuterVolumeSpecName: "kube-api-access-75g78") pod "0a3adf3c-282f-4c72-97a0-343b8b132f5b" (UID: "0a3adf3c-282f-4c72-97a0-343b8b132f5b"). InnerVolumeSpecName "kube-api-access-75g78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:51:48.813564 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.813526    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75g78\" (UniqueName: \"kubernetes.io/projected/0a3adf3c-282f-4c72-97a0-343b8b132f5b-kube-api-access-75g78\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:51:48.813564 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:48.813565    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a3adf3c-282f-4c72-97a0-343b8b132f5b-extensions-socket-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:51:49.345846 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:49.345817    2569 generic.go:356] "Generic (PLEG): container finished" podID="0a3adf3c-282f-4c72-97a0-343b8b132f5b" containerID="2dace548b3b4fdcf6519a4b7e4ec75e754fd0332f18e70c23a0a19f546d8ffb3" exitCode=2
Jul 30 03:51:49.346049 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:49.345860    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"
Jul 30 03:51:49.346049 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:49.345914    2569 scope.go:117] "RemoveContainer" containerID="2dace548b3b4fdcf6519a4b7e4ec75e754fd0332f18e70c23a0a19f546d8ffb3"
Jul 30 03:51:49.347915 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:51:49.347886    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sghmh\" is forbidden: User \"system:node:ip-10-0-136-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-104.ec2.internal' and this object" podUID="0a3adf3c-282f-4c72-97a0-343b8b132f5b" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"
Jul 30 03:51:49.355148 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:49.355134    2569 scope.go:117] "RemoveContainer" containerID="2dace548b3b4fdcf6519a4b7e4ec75e754fd0332f18e70c23a0a19f546d8ffb3"
Jul 30 03:51:49.355416 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:51:49.355394    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dace548b3b4fdcf6519a4b7e4ec75e754fd0332f18e70c23a0a19f546d8ffb3\": container with ID starting with 2dace548b3b4fdcf6519a4b7e4ec75e754fd0332f18e70c23a0a19f546d8ffb3 not found: ID does not exist" containerID="2dace548b3b4fdcf6519a4b7e4ec75e754fd0332f18e70c23a0a19f546d8ffb3"
Jul 30 03:51:49.355464 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:49.355429    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dace548b3b4fdcf6519a4b7e4ec75e754fd0332f18e70c23a0a19f546d8ffb3"} err="failed to get container status \"2dace548b3b4fdcf6519a4b7e4ec75e754fd0332f18e70c23a0a19f546d8ffb3\": rpc error: code = NotFound desc = could not find container \"2dace548b3b4fdcf6519a4b7e4ec75e754fd0332f18e70c23a0a19f546d8ffb3\": container with ID starting with 2dace548b3b4fdcf6519a4b7e4ec75e754fd0332f18e70c23a0a19f546d8ffb3 not found: ID does not exist"
Jul 30 03:51:49.355751 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:51:49.355733    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sghmh\" is forbidden: User \"system:node:ip-10-0-136-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-104.ec2.internal' and this object" podUID="0a3adf3c-282f-4c72-97a0-343b8b132f5b" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sghmh"
Jul 30 03:51:49.464366 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:49.464342    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a3adf3c-282f-4c72-97a0-343b8b132f5b" path="/var/lib/kubelet/pods/0a3adf3c-282f-4c72-97a0-343b8b132f5b/volumes"
Jul 30 03:51:52.244630 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:52.244589    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:52.344156 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:52.344123    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:56.297099 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:56.297064    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:56.358931 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:56.358906    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:58.047480 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:58.047444    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:58.047918 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:58.047594    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:58.052236 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:58.052210    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:58.397909 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:58.397827    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6868887c67-dklsc"
Jul 30 03:51:58.446549 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:58.446505    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5db69bb768-wc6c8"]
Jul 30 03:51:58.555037 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:58.554994    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:58.604451 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:58.604420    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:59.348142 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:51:59.348107    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"
Jul 30 03:52:05.899017 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:05.898981    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:06.005571 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:06.005517    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:09.294958 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:09.294918    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:09.401264 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:09.401231    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:11.705965 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:11.705924    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"]
Jul 30 03:52:11.706380 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:11.706138    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp" podUID="3a509a2d-bed0-452c-baf8-ea326bd2c52c" containerName="manager" containerID="cri-o://77100b65ce500ae6c6dab8c2fa7e6a2feec6b6f3b005dd55779459f48d987a55" gracePeriod=10
Jul 30 03:52:11.954782 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:11.954759    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"
Jul 30 03:52:12.085487 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.085457    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bh4qt\" (UniqueName: \"kubernetes.io/projected/3a509a2d-bed0-452c-baf8-ea326bd2c52c-kube-api-access-bh4qt\") pod \"3a509a2d-bed0-452c-baf8-ea326bd2c52c\" (UID: \"3a509a2d-bed0-452c-baf8-ea326bd2c52c\") "
Jul 30 03:52:12.085675 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.085552    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a509a2d-bed0-452c-baf8-ea326bd2c52c-extensions-socket-volume\") pod \"3a509a2d-bed0-452c-baf8-ea326bd2c52c\" (UID: \"3a509a2d-bed0-452c-baf8-ea326bd2c52c\") "
Jul 30 03:52:12.085905 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.085877    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a509a2d-bed0-452c-baf8-ea326bd2c52c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3a509a2d-bed0-452c-baf8-ea326bd2c52c" (UID: "3a509a2d-bed0-452c-baf8-ea326bd2c52c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:52:12.087675 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.087650    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a509a2d-bed0-452c-baf8-ea326bd2c52c-kube-api-access-bh4qt" (OuterVolumeSpecName: "kube-api-access-bh4qt") pod "3a509a2d-bed0-452c-baf8-ea326bd2c52c" (UID: "3a509a2d-bed0-452c-baf8-ea326bd2c52c"). InnerVolumeSpecName "kube-api-access-bh4qt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:52:12.186775 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.186745    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh4qt\" (UniqueName: \"kubernetes.io/projected/3a509a2d-bed0-452c-baf8-ea326bd2c52c-kube-api-access-bh4qt\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:52:12.186775 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.186769    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a509a2d-bed0-452c-baf8-ea326bd2c52c-extensions-socket-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:52:12.462970 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.462891    2569 generic.go:356] "Generic (PLEG): container finished" podID="3a509a2d-bed0-452c-baf8-ea326bd2c52c" containerID="77100b65ce500ae6c6dab8c2fa7e6a2feec6b6f3b005dd55779459f48d987a55" exitCode=0
Jul 30 03:52:12.462970 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.462932    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"
Jul 30 03:52:12.463156 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.462974    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp" event={"ID":"3a509a2d-bed0-452c-baf8-ea326bd2c52c","Type":"ContainerDied","Data":"77100b65ce500ae6c6dab8c2fa7e6a2feec6b6f3b005dd55779459f48d987a55"}
Jul 30 03:52:12.463156 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.463017    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp" event={"ID":"3a509a2d-bed0-452c-baf8-ea326bd2c52c","Type":"ContainerDied","Data":"70aa72c0ff5641f0eb2752da7024254e2ce2dbbeade5650fc403a362158c2dd3"}
Jul 30 03:52:12.463156 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.463039    2569 scope.go:117] "RemoveContainer" containerID="77100b65ce500ae6c6dab8c2fa7e6a2feec6b6f3b005dd55779459f48d987a55"
Jul 30 03:52:12.475334 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.475310    2569 scope.go:117] "RemoveContainer" containerID="77100b65ce500ae6c6dab8c2fa7e6a2feec6b6f3b005dd55779459f48d987a55"
Jul 30 03:52:12.475618 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:52:12.475600    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77100b65ce500ae6c6dab8c2fa7e6a2feec6b6f3b005dd55779459f48d987a55\": container with ID starting with 77100b65ce500ae6c6dab8c2fa7e6a2feec6b6f3b005dd55779459f48d987a55 not found: ID does not exist" containerID="77100b65ce500ae6c6dab8c2fa7e6a2feec6b6f3b005dd55779459f48d987a55"
Jul 30 03:52:12.475698 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.475628    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77100b65ce500ae6c6dab8c2fa7e6a2feec6b6f3b005dd55779459f48d987a55"} err="failed to get container status \"77100b65ce500ae6c6dab8c2fa7e6a2feec6b6f3b005dd55779459f48d987a55\": rpc error: code = NotFound desc = could not find container \"77100b65ce500ae6c6dab8c2fa7e6a2feec6b6f3b005dd55779459f48d987a55\": container with ID starting with 77100b65ce500ae6c6dab8c2fa7e6a2feec6b6f3b005dd55779459f48d987a55 not found: ID does not exist"
Jul 30 03:52:12.492202 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.492180    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"]
Jul 30 03:52:12.496644 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:12.496625    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-l2hpp"]
Jul 30 03:52:13.469723 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:13.469687    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a509a2d-bed0-452c-baf8-ea326bd2c52c" path="/var/lib/kubelet/pods/3a509a2d-bed0-452c-baf8-ea326bd2c52c/volumes"
Jul 30 03:52:14.093350 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:14.093316    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:14.143507 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:14.143431    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:14.687396 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:14.687365    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:14.787754 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:14.787724    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:15.779972 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:15.779942    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:16.261475 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:16.261374    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:18.491591 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:18.491556    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:18.546978 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:18.546948    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:23.467134 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.467053    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5db69bb768-wc6c8" podUID="fcbc016d-220a-40ad-b991-6f446d37fdb9" containerName="console" containerID="cri-o://19c471238f0e3567777634972945817fde8ab115e4cdbf015025d10a4b45375e" gracePeriod=15
Jul 30 03:52:23.703802 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.703781    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:52:23.878972 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.878944    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-oauth-config\") pod \"fcbc016d-220a-40ad-b991-6f446d37fdb9\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") "
Jul 30 03:52:23.879129 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.878985    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-config\") pod \"fcbc016d-220a-40ad-b991-6f446d37fdb9\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") "
Jul 30 03:52:23.879129 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.879012    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-oauth-serving-cert\") pod \"fcbc016d-220a-40ad-b991-6f446d37fdb9\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") "
Jul 30 03:52:23.879226 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.879133    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znf9c\" (UniqueName: \"kubernetes.io/projected/fcbc016d-220a-40ad-b991-6f446d37fdb9-kube-api-access-znf9c\") pod \"fcbc016d-220a-40ad-b991-6f446d37fdb9\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") "
Jul 30 03:52:23.879282 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.879253    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-trusted-ca-bundle\") pod \"fcbc016d-220a-40ad-b991-6f446d37fdb9\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") "
Jul 30 03:52:23.879337 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.879284    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-serving-cert\") pod \"fcbc016d-220a-40ad-b991-6f446d37fdb9\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") "
Jul 30 03:52:23.879337 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.879320    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-service-ca\") pod \"fcbc016d-220a-40ad-b991-6f446d37fdb9\" (UID: \"fcbc016d-220a-40ad-b991-6f446d37fdb9\") "
Jul 30 03:52:23.879441 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.879385    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fcbc016d-220a-40ad-b991-6f446d37fdb9" (UID: "fcbc016d-220a-40ad-b991-6f446d37fdb9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:52:23.879441 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.879383    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-config" (OuterVolumeSpecName: "console-config") pod "fcbc016d-220a-40ad-b991-6f446d37fdb9" (UID: "fcbc016d-220a-40ad-b991-6f446d37fdb9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:52:23.879615 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.879591    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-config\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:52:23.879615 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.879618    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-oauth-serving-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:52:23.879802 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.879690    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fcbc016d-220a-40ad-b991-6f446d37fdb9" (UID: "fcbc016d-220a-40ad-b991-6f446d37fdb9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:52:23.880025 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.879829    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-service-ca" (OuterVolumeSpecName: "service-ca") pod "fcbc016d-220a-40ad-b991-6f446d37fdb9" (UID: "fcbc016d-220a-40ad-b991-6f446d37fdb9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:52:23.881365 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.881341    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcbc016d-220a-40ad-b991-6f446d37fdb9-kube-api-access-znf9c" (OuterVolumeSpecName: "kube-api-access-znf9c") pod "fcbc016d-220a-40ad-b991-6f446d37fdb9" (UID: "fcbc016d-220a-40ad-b991-6f446d37fdb9"). InnerVolumeSpecName "kube-api-access-znf9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:52:23.881451 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.881346    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fcbc016d-220a-40ad-b991-6f446d37fdb9" (UID: "fcbc016d-220a-40ad-b991-6f446d37fdb9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:52:23.881451 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.881389    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fcbc016d-220a-40ad-b991-6f446d37fdb9" (UID: "fcbc016d-220a-40ad-b991-6f446d37fdb9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:52:23.980966 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.980939    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znf9c\" (UniqueName: \"kubernetes.io/projected/fcbc016d-220a-40ad-b991-6f446d37fdb9-kube-api-access-znf9c\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:52:23.980966 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.980964    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-trusted-ca-bundle\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:52:23.981330 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.980975    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-serving-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:52:23.981330 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.980990    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcbc016d-220a-40ad-b991-6f446d37fdb9-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:52:23.981330 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:23.981004    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcbc016d-220a-40ad-b991-6f446d37fdb9-console-oauth-config\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:52:24.523117 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:24.523085    2569 generic.go:356] "Generic (PLEG): container finished" podID="fcbc016d-220a-40ad-b991-6f446d37fdb9" containerID="19c471238f0e3567777634972945817fde8ab115e4cdbf015025d10a4b45375e" exitCode=0
Jul 30 03:52:24.523521 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:24.523127    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db69bb768-wc6c8"
Jul 30 03:52:24.523521 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:24.523152    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db69bb768-wc6c8" event={"ID":"fcbc016d-220a-40ad-b991-6f446d37fdb9","Type":"ContainerDied","Data":"19c471238f0e3567777634972945817fde8ab115e4cdbf015025d10a4b45375e"}
Jul 30 03:52:24.523521 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:24.523191    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db69bb768-wc6c8" event={"ID":"fcbc016d-220a-40ad-b991-6f446d37fdb9","Type":"ContainerDied","Data":"dc94118bfa5f3e60934184ffe367f5957ce8d0b9e4477a76b8eb7c442979be3f"}
Jul 30 03:52:24.523521 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:24.523209    2569 scope.go:117] "RemoveContainer" containerID="19c471238f0e3567777634972945817fde8ab115e4cdbf015025d10a4b45375e"
Jul 30 03:52:24.536684 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:24.536665    2569 scope.go:117] "RemoveContainer" containerID="19c471238f0e3567777634972945817fde8ab115e4cdbf015025d10a4b45375e"
Jul 30 03:52:24.536948 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:52:24.536930    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19c471238f0e3567777634972945817fde8ab115e4cdbf015025d10a4b45375e\": container with ID starting with 19c471238f0e3567777634972945817fde8ab115e4cdbf015025d10a4b45375e not found: ID does not exist" containerID="19c471238f0e3567777634972945817fde8ab115e4cdbf015025d10a4b45375e"
Jul 30 03:52:24.537022 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:24.536956    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19c471238f0e3567777634972945817fde8ab115e4cdbf015025d10a4b45375e"} err="failed to get container status \"19c471238f0e3567777634972945817fde8ab115e4cdbf015025d10a4b45375e\": rpc error: code = NotFound desc = could not find container \"19c471238f0e3567777634972945817fde8ab115e4cdbf015025d10a4b45375e\": container with ID starting with 19c471238f0e3567777634972945817fde8ab115e4cdbf015025d10a4b45375e not found: ID does not exist"
Jul 30 03:52:24.546183 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:24.546160    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5db69bb768-wc6c8"]
Jul 30 03:52:24.549646 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:24.549623    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5db69bb768-wc6c8"]
Jul 30 03:52:25.465743 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:25.465703    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcbc016d-220a-40ad-b991-6f446d37fdb9" path="/var/lib/kubelet/pods/fcbc016d-220a-40ad-b991-6f446d37fdb9/volumes"
Jul 30 03:52:31.044053 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.044021    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"]
Jul 30 03:52:31.047193 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.044776    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a509a2d-bed0-452c-baf8-ea326bd2c52c" containerName="manager"
Jul 30 03:52:31.047193 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.044810    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a509a2d-bed0-452c-baf8-ea326bd2c52c" containerName="manager"
Jul 30 03:52:31.047193 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.044866    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcbc016d-220a-40ad-b991-6f446d37fdb9" containerName="console"
Jul 30 03:52:31.047193 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.044875    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcbc016d-220a-40ad-b991-6f446d37fdb9" containerName="console"
Jul 30 03:52:31.047193 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.045078    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcbc016d-220a-40ad-b991-6f446d37fdb9" containerName="console"
Jul 30 03:52:31.047193 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.045102    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a509a2d-bed0-452c-baf8-ea326bd2c52c" containerName="manager"
Jul 30 03:52:31.050145 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.048479    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"
Jul 30 03:52:31.050771 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.050753    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 03:52:31.050887 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.050795    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7jnsd\""
Jul 30 03:52:31.051408 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.051392    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 03:52:31.051500 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.051421    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 03:52:31.051814 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.051790    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:31.059996 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.059976    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"]
Jul 30 03:52:31.148667 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.148637    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:31.150378 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.150355    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"]
Jul 30 03:52:31.238787 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.238757    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/993e5595-f67f-4322-8348-267cb10466e6-config-file\") pod \"limitador-limitador-64c94fd64f-sqhtb\" (UID: \"993e5595-f67f-4322-8348-267cb10466e6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"
Jul 30 03:52:31.238787 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.238786    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfchc\" (UniqueName: \"kubernetes.io/projected/993e5595-f67f-4322-8348-267cb10466e6-kube-api-access-nfchc\") pod \"limitador-limitador-64c94fd64f-sqhtb\" (UID: \"993e5595-f67f-4322-8348-267cb10466e6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"
Jul 30 03:52:31.250450 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.250425    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:31.339487 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.339420    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/993e5595-f67f-4322-8348-267cb10466e6-config-file\") pod \"limitador-limitador-64c94fd64f-sqhtb\" (UID: \"993e5595-f67f-4322-8348-267cb10466e6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"
Jul 30 03:52:31.339487 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.339457    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfchc\" (UniqueName: \"kubernetes.io/projected/993e5595-f67f-4322-8348-267cb10466e6-kube-api-access-nfchc\") pod \"limitador-limitador-64c94fd64f-sqhtb\" (UID: \"993e5595-f67f-4322-8348-267cb10466e6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"
Jul 30 03:52:31.340016 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.339998    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/993e5595-f67f-4322-8348-267cb10466e6-config-file\") pod \"limitador-limitador-64c94fd64f-sqhtb\" (UID: \"993e5595-f67f-4322-8348-267cb10466e6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"
Jul 30 03:52:31.346811 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.346795    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfchc\" (UniqueName: \"kubernetes.io/projected/993e5595-f67f-4322-8348-267cb10466e6-kube-api-access-nfchc\") pod \"limitador-limitador-64c94fd64f-sqhtb\" (UID: \"993e5595-f67f-4322-8348-267cb10466e6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"
Jul 30 03:52:31.360685 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.360663    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"
Jul 30 03:52:31.489940 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.489918    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"]
Jul 30 03:52:31.492181 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:52:31.492152    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod993e5595_f67f_4322_8348_267cb10466e6.slice/crio-29901beefc17cee10bc43d5cdab076fc8daf02a533da043444db2b55bfb3948f WatchSource:0}: Error finding container 29901beefc17cee10bc43d5cdab076fc8daf02a533da043444db2b55bfb3948f: Status 404 returned error can't find the container with id 29901beefc17cee10bc43d5cdab076fc8daf02a533da043444db2b55bfb3948f
Jul 30 03:52:31.556391 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.556356    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb" event={"ID":"993e5595-f67f-4322-8348-267cb10466e6","Type":"ContainerStarted","Data":"29901beefc17cee10bc43d5cdab076fc8daf02a533da043444db2b55bfb3948f"}
Jul 30 03:52:31.840346 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.840312    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-7l8d9"]
Jul 30 03:52:31.843216 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.843193    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7l8d9"
Jul 30 03:52:31.845183 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.845160    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lzq79\""
Jul 30 03:52:31.851479 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.851460    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7l8d9"]
Jul 30 03:52:31.879629 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.879591    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:31.934192 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.934167    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:31.944252 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:31.944229    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tprlk\" (UniqueName: \"kubernetes.io/projected/f7205adc-efed-412f-adbb-5eafe870465a-kube-api-access-tprlk\") pod \"authorino-58cd955675-7l8d9\" (UID: \"f7205adc-efed-412f-adbb-5eafe870465a\") " pod="kuadrant-system/authorino-58cd955675-7l8d9"
Jul 30 03:52:32.045297 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:32.045244    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tprlk\" (UniqueName: \"kubernetes.io/projected/f7205adc-efed-412f-adbb-5eafe870465a-kube-api-access-tprlk\") pod \"authorino-58cd955675-7l8d9\" (UID: \"f7205adc-efed-412f-adbb-5eafe870465a\") " pod="kuadrant-system/authorino-58cd955675-7l8d9"
Jul 30 03:52:32.052907 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:32.052883    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tprlk\" (UniqueName: \"kubernetes.io/projected/f7205adc-efed-412f-adbb-5eafe870465a-kube-api-access-tprlk\") pod \"authorino-58cd955675-7l8d9\" (UID: \"f7205adc-efed-412f-adbb-5eafe870465a\") " pod="kuadrant-system/authorino-58cd955675-7l8d9"
Jul 30 03:52:32.154153 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:32.154073    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7l8d9"
Jul 30 03:52:32.331303 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:32.331230    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7l8d9"]
Jul 30 03:52:32.334839 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:52:32.334811    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7205adc_efed_412f_adbb_5eafe870465a.slice/crio-9927ad1756746a66e022b265c4f9101ae24c29ad2c09074798295027738cfffa WatchSource:0}: Error finding container 9927ad1756746a66e022b265c4f9101ae24c29ad2c09074798295027738cfffa: Status 404 returned error can't find the container with id 9927ad1756746a66e022b265c4f9101ae24c29ad2c09074798295027738cfffa
Jul 30 03:52:32.562715 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:32.562677    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7l8d9" event={"ID":"f7205adc-efed-412f-adbb-5eafe870465a","Type":"ContainerStarted","Data":"9927ad1756746a66e022b265c4f9101ae24c29ad2c09074798295027738cfffa"}
Jul 30 03:52:34.576655 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:34.576615    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb" event={"ID":"993e5595-f67f-4322-8348-267cb10466e6","Type":"ContainerStarted","Data":"63c2dc808a4c156721d414483b2ee69963c6bbfb001d1127396de0a04019e1ec"}
Jul 30 03:52:34.577065 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:34.576709    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"
Jul 30 03:52:34.593312 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:34.593271    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb" podStartSLOduration=0.916013267 podStartE2EDuration="3.59325818s" totalImagesPullingTime="2.677244913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:52:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:52:34.59140566 +0000 UTC m=+453.630116682" watchObservedRunningTime="2026-07-30 03:52:34.59325818 +0000 UTC m=+453.631969198"
Jul 30 03:52:35.116060 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:52:35.116018    2569 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 500 Internal Server Error; artifact err: provided artifact is a container image" image="quay.io/kuadrant/authorino:v0.24.0"
Jul 30 03:52:35.116193 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:52:35.116127    2569 kuberuntime_manager.go:1459] "Unhandled Error" err="container authorino start failed in pod authorino-58cd955675-7l8d9_kuadrant-system(f7205adc-efed-412f-adbb-5eafe870465a): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 500 Internal Server Error; artifact err: provided artifact is a container image" logger="UnhandledError"
Jul 30 03:52:35.116193 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:52:35.116159    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"authorino\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 500 Internal Server Error; artifact err: provided artifact is a container image\"" pod="kuadrant-system/authorino-58cd955675-7l8d9" podUID="f7205adc-efed-412f-adbb-5eafe870465a"
Jul 30 03:52:35.582431 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:52:35.582395    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"authorino\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/kuadrant/authorino:v0.24.0\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 500 Internal Server Error; artifact err: provided artifact is a container image\"" pod="kuadrant-system/authorino-58cd955675-7l8d9" podUID="f7205adc-efed-412f-adbb-5eafe870465a"
Jul 30 03:52:36.871685 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:36.871649    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7l8d9"]
Jul 30 03:52:36.997880 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:36.997848    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7l8d9"
Jul 30 03:52:37.085940 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:37.085911    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tprlk\" (UniqueName: \"kubernetes.io/projected/f7205adc-efed-412f-adbb-5eafe870465a-kube-api-access-tprlk\") pod \"f7205adc-efed-412f-adbb-5eafe870465a\" (UID: \"f7205adc-efed-412f-adbb-5eafe870465a\") "
Jul 30 03:52:37.088050 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:37.088021    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7205adc-efed-412f-adbb-5eafe870465a-kube-api-access-tprlk" (OuterVolumeSpecName: "kube-api-access-tprlk") pod "f7205adc-efed-412f-adbb-5eafe870465a" (UID: "f7205adc-efed-412f-adbb-5eafe870465a"). InnerVolumeSpecName "kube-api-access-tprlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:52:37.186501 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:37.186421    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tprlk\" (UniqueName: \"kubernetes.io/projected/f7205adc-efed-412f-adbb-5eafe870465a-kube-api-access-tprlk\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:52:37.591753 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:37.591719    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7l8d9" event={"ID":"f7205adc-efed-412f-adbb-5eafe870465a","Type":"ContainerDied","Data":"9927ad1756746a66e022b265c4f9101ae24c29ad2c09074798295027738cfffa"}
Jul 30 03:52:37.591902 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:37.591801    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7l8d9"
Jul 30 03:52:37.615814 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:37.615771    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7l8d9"]
Jul 30 03:52:37.617499 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:37.617479    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-7l8d9"]
Jul 30 03:52:39.464643 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:39.464610    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7205adc-efed-412f-adbb-5eafe870465a" path="/var/lib/kubelet/pods/f7205adc-efed-412f-adbb-5eafe870465a/volumes"
Jul 30 03:52:45.583293 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:45.583261    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"
Jul 30 03:52:47.880370 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:47.880338    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"]
Jul 30 03:52:47.880843 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:47.880505    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb" podUID="993e5595-f67f-4322-8348-267cb10466e6" containerName="limitador" containerID="cri-o://63c2dc808a4c156721d414483b2ee69963c6bbfb001d1127396de0a04019e1ec" gracePeriod=30
Jul 30 03:52:48.424882 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.424859    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"
Jul 30 03:52:48.456890 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.456833    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfchc\" (UniqueName: \"kubernetes.io/projected/993e5595-f67f-4322-8348-267cb10466e6-kube-api-access-nfchc\") pod \"993e5595-f67f-4322-8348-267cb10466e6\" (UID: \"993e5595-f67f-4322-8348-267cb10466e6\") "
Jul 30 03:52:48.456890 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.456877    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/993e5595-f67f-4322-8348-267cb10466e6-config-file\") pod \"993e5595-f67f-4322-8348-267cb10466e6\" (UID: \"993e5595-f67f-4322-8348-267cb10466e6\") "
Jul 30 03:52:48.457138 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.457114    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/993e5595-f67f-4322-8348-267cb10466e6-config-file" (OuterVolumeSpecName: "config-file") pod "993e5595-f67f-4322-8348-267cb10466e6" (UID: "993e5595-f67f-4322-8348-267cb10466e6"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:52:48.458859 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.458837    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/993e5595-f67f-4322-8348-267cb10466e6-kube-api-access-nfchc" (OuterVolumeSpecName: "kube-api-access-nfchc") pod "993e5595-f67f-4322-8348-267cb10466e6" (UID: "993e5595-f67f-4322-8348-267cb10466e6"). InnerVolumeSpecName "kube-api-access-nfchc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:52:48.557746 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.557723    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfchc\" (UniqueName: \"kubernetes.io/projected/993e5595-f67f-4322-8348-267cb10466e6-kube-api-access-nfchc\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:52:48.557842 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.557748    2569 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/993e5595-f67f-4322-8348-267cb10466e6-config-file\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:52:48.645735 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.645704    2569 generic.go:356] "Generic (PLEG): container finished" podID="993e5595-f67f-4322-8348-267cb10466e6" containerID="63c2dc808a4c156721d414483b2ee69963c6bbfb001d1127396de0a04019e1ec" exitCode=0
Jul 30 03:52:48.645868 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.645740    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"
Jul 30 03:52:48.645868 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.645789    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb" event={"ID":"993e5595-f67f-4322-8348-267cb10466e6","Type":"ContainerDied","Data":"63c2dc808a4c156721d414483b2ee69963c6bbfb001d1127396de0a04019e1ec"}
Jul 30 03:52:48.645868 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.645850    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sqhtb" event={"ID":"993e5595-f67f-4322-8348-267cb10466e6","Type":"ContainerDied","Data":"29901beefc17cee10bc43d5cdab076fc8daf02a533da043444db2b55bfb3948f"}
Jul 30 03:52:48.645991 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.645883    2569 scope.go:117] "RemoveContainer" containerID="63c2dc808a4c156721d414483b2ee69963c6bbfb001d1127396de0a04019e1ec"
Jul 30 03:52:48.654825 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.654811    2569 scope.go:117] "RemoveContainer" containerID="63c2dc808a4c156721d414483b2ee69963c6bbfb001d1127396de0a04019e1ec"
Jul 30 03:52:48.655069 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:52:48.655052    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63c2dc808a4c156721d414483b2ee69963c6bbfb001d1127396de0a04019e1ec\": container with ID starting with 63c2dc808a4c156721d414483b2ee69963c6bbfb001d1127396de0a04019e1ec not found: ID does not exist" containerID="63c2dc808a4c156721d414483b2ee69963c6bbfb001d1127396de0a04019e1ec"
Jul 30 03:52:48.655111 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.655076    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63c2dc808a4c156721d414483b2ee69963c6bbfb001d1127396de0a04019e1ec"} err="failed to get container status \"63c2dc808a4c156721d414483b2ee69963c6bbfb001d1127396de0a04019e1ec\": rpc error: code = NotFound desc = could not find container \"63c2dc808a4c156721d414483b2ee69963c6bbfb001d1127396de0a04019e1ec\": container with ID starting with 63c2dc808a4c156721d414483b2ee69963c6bbfb001d1127396de0a04019e1ec not found: ID does not exist"
Jul 30 03:52:48.665838 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.665818    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"]
Jul 30 03:52:48.669512 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:48.669492    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sqhtb"]
Jul 30 03:52:49.254034 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.253994    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:49.353329 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.353292    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:49.465309 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.465277    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="993e5595-f67f-4322-8348-267cb10466e6" path="/var/lib/kubelet/pods/993e5595-f67f-4322-8348-267cb10466e6/volumes"
Jul 30 03:52:49.688598 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.688568    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd"]
Jul 30 03:52:49.688920 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.688903    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="993e5595-f67f-4322-8348-267cb10466e6" containerName="limitador"
Jul 30 03:52:49.688920 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.688921    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="993e5595-f67f-4322-8348-267cb10466e6" containerName="limitador"
Jul 30 03:52:49.689043 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.688975    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="993e5595-f67f-4322-8348-267cb10466e6" containerName="limitador"
Jul 30 03:52:49.693339 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.693318    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd"
Jul 30 03:52:49.695443 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.695425    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 30 03:52:49.695995 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.695975    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 03:52:49.696103 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.695983    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-drkh6\""
Jul 30 03:52:49.696103 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.695999    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 03:52:49.699319 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.699299    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd"]
Jul 30 03:52:49.727884 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.727848    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:49.767091 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.767064    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tck8w\" (UniqueName: \"kubernetes.io/projected/6b0117bb-3d81-4a0b-bc50-c72fd1585c99-kube-api-access-tck8w\") pod \"postgres-69fb4594cf-bpzjd\" (UID: \"6b0117bb-3d81-4a0b-bc50-c72fd1585c99\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd"
Jul 30 03:52:49.767206 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.767141    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6b0117bb-3d81-4a0b-bc50-c72fd1585c99-data\") pod \"postgres-69fb4594cf-bpzjd\" (UID: \"6b0117bb-3d81-4a0b-bc50-c72fd1585c99\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd"
Jul 30 03:52:49.868221 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.868194    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6b0117bb-3d81-4a0b-bc50-c72fd1585c99-data\") pod \"postgres-69fb4594cf-bpzjd\" (UID: \"6b0117bb-3d81-4a0b-bc50-c72fd1585c99\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd"
Jul 30 03:52:49.868341 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.868247    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tck8w\" (UniqueName: \"kubernetes.io/projected/6b0117bb-3d81-4a0b-bc50-c72fd1585c99-kube-api-access-tck8w\") pod \"postgres-69fb4594cf-bpzjd\" (UID: \"6b0117bb-3d81-4a0b-bc50-c72fd1585c99\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd"
Jul 30 03:52:49.868597 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.868578    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6b0117bb-3d81-4a0b-bc50-c72fd1585c99-data\") pod \"postgres-69fb4594cf-bpzjd\" (UID: \"6b0117bb-3d81-4a0b-bc50-c72fd1585c99\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd"
Jul 30 03:52:49.876030 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:49.876010    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tck8w\" (UniqueName: \"kubernetes.io/projected/6b0117bb-3d81-4a0b-bc50-c72fd1585c99-kube-api-access-tck8w\") pod \"postgres-69fb4594cf-bpzjd\" (UID: \"6b0117bb-3d81-4a0b-bc50-c72fd1585c99\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd"
Jul 30 03:52:50.005461 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:50.005392    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd"
Jul 30 03:52:50.134026 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:50.134000    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd"]
Jul 30 03:52:50.135972 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:52:50.135948    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b0117bb_3d81_4a0b_bc50_c72fd1585c99.slice/crio-89385f3bfaada7608748d51833cba4e41a9fea2cf139571abb3d2d09ffd6f617 WatchSource:0}: Error finding container 89385f3bfaada7608748d51833cba4e41a9fea2cf139571abb3d2d09ffd6f617: Status 404 returned error can't find the container with id 89385f3bfaada7608748d51833cba4e41a9fea2cf139571abb3d2d09ffd6f617
Jul 30 03:52:50.657873 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:50.657840    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd" event={"ID":"6b0117bb-3d81-4a0b-bc50-c72fd1585c99","Type":"ContainerStarted","Data":"89385f3bfaada7608748d51833cba4e41a9fea2cf139571abb3d2d09ffd6f617"}
Jul 30 03:52:55.687439 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:55.687400    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd" event={"ID":"6b0117bb-3d81-4a0b-bc50-c72fd1585c99","Type":"ContainerStarted","Data":"01fb40b7666dd3c87a9031058ed4e25b22e6a85f40812dc10d18e4ae0a4f9f61"}
Jul 30 03:52:55.687849 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:55.687523    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd"
Jul 30 03:52:55.725949 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:52:55.725906    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd" podStartSLOduration=1.694969704 podStartE2EDuration="6.72589097s" totalImagesPullingTime="5.030921266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:52:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:52:55.714923567 +0000 UTC m=+474.753634589" watchObservedRunningTime="2026-07-30 03:52:55.72589097 +0000 UTC m=+474.764602044"
Jul 30 03:53:01.720507 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:01.720430    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bpzjd"
Jul 30 03:53:02.047686 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:02.047651    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:02.481856 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:02.481749    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:08.615194 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.615163    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:08.628093 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.628063    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68d9fb7cc7-nwhgm"]
Jul 30 03:53:08.652135 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.652109    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68d9fb7cc7-nwhgm"]
Jul 30 03:53:08.652288 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.652197    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm"
Jul 30 03:53:08.654614 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.654592    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 03:53:08.654852 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.654601    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 03:53:08.654949 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.654610    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ld5c2\""
Jul 30 03:53:08.697930 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.697906    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:08.702911 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.702886    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8gfd\" (UniqueName: \"kubernetes.io/projected/d79c28c2-2ec8-4194-8939-27af384d4936-kube-api-access-z8gfd\") pod \"maas-controller-68d9fb7cc7-nwhgm\" (UID: \"d79c28c2-2ec8-4194-8939-27af384d4936\") " pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm"
Jul 30 03:53:08.703009 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.702932    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d79c28c2-2ec8-4194-8939-27af384d4936-webhook-cert\") pod \"maas-controller-68d9fb7cc7-nwhgm\" (UID: \"d79c28c2-2ec8-4194-8939-27af384d4936\") " pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm"
Jul 30 03:53:08.762667 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.762638    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:08.803283 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.803259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8gfd\" (UniqueName: \"kubernetes.io/projected/d79c28c2-2ec8-4194-8939-27af384d4936-kube-api-access-z8gfd\") pod \"maas-controller-68d9fb7cc7-nwhgm\" (UID: \"d79c28c2-2ec8-4194-8939-27af384d4936\") " pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm"
Jul 30 03:53:08.803416 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.803334    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d79c28c2-2ec8-4194-8939-27af384d4936-webhook-cert\") pod \"maas-controller-68d9fb7cc7-nwhgm\" (UID: \"d79c28c2-2ec8-4194-8939-27af384d4936\") " pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm"
Jul 30 03:53:08.806200 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.806179    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d79c28c2-2ec8-4194-8939-27af384d4936-webhook-cert\") pod \"maas-controller-68d9fb7cc7-nwhgm\" (UID: \"d79c28c2-2ec8-4194-8939-27af384d4936\") " pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm"
Jul 30 03:53:08.814074 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.814038    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8gfd\" (UniqueName: \"kubernetes.io/projected/d79c28c2-2ec8-4194-8939-27af384d4936-kube-api-access-z8gfd\") pod \"maas-controller-68d9fb7cc7-nwhgm\" (UID: \"d79c28c2-2ec8-4194-8939-27af384d4936\") " pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm"
Jul 30 03:53:08.963317 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:08.963244    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm"
Jul 30 03:53:09.100514 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:09.098685    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68d9fb7cc7-nwhgm"]
Jul 30 03:53:09.102093 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:53:09.102062    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd79c28c2_2ec8_4194_8939_27af384d4936.slice/crio-8124c3d639a2c290616b77c7bd614824b09840d1659762d90da1ad130f94c736 WatchSource:0}: Error finding container 8124c3d639a2c290616b77c7bd614824b09840d1659762d90da1ad130f94c736: Status 404 returned error can't find the container with id 8124c3d639a2c290616b77c7bd614824b09840d1659762d90da1ad130f94c736
Jul 30 03:53:09.756182 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:09.756141    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm" event={"ID":"d79c28c2-2ec8-4194-8939-27af384d4936","Type":"ContainerStarted","Data":"8124c3d639a2c290616b77c7bd614824b09840d1659762d90da1ad130f94c736"}
Jul 30 03:53:11.500602 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:11.500562    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:11.602794 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:11.602760    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:11.767285 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:11.767242    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm" event={"ID":"d79c28c2-2ec8-4194-8939-27af384d4936","Type":"ContainerStarted","Data":"4d5f965364606bc28910b8c3196df704c65b7c53d296ded3101379ea32784bdc"}
Jul 30 03:53:11.767450 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:11.767432    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm"
Jul 30 03:53:11.784960 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:11.784913    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm" podStartSLOduration=1.532412095 podStartE2EDuration="3.784895576s" totalImagesPullingTime="2.252483481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:53:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:53:11.783628155 +0000 UTC m=+490.822339176" watchObservedRunningTime="2026-07-30 03:53:11.784895576 +0000 UTC m=+490.823606596"
Jul 30 03:53:22.775457 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:22.775422    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm"
Jul 30 03:53:23.025034 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.024999    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68d9fb7cc7-nwhgm"]
Jul 30 03:53:23.025192 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.025166    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm" podUID="d79c28c2-2ec8-4194-8939-27af384d4936" containerName="manager" containerID="cri-o://4d5f965364606bc28910b8c3196df704c65b7c53d296ded3101379ea32784bdc" gracePeriod=30
Jul 30 03:53:23.272336 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.272307    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm"
Jul 30 03:53:23.316712 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.316649    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8gfd\" (UniqueName: \"kubernetes.io/projected/d79c28c2-2ec8-4194-8939-27af384d4936-kube-api-access-z8gfd\") pod \"d79c28c2-2ec8-4194-8939-27af384d4936\" (UID: \"d79c28c2-2ec8-4194-8939-27af384d4936\") "
Jul 30 03:53:23.316712 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.316682    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d79c28c2-2ec8-4194-8939-27af384d4936-webhook-cert\") pod \"d79c28c2-2ec8-4194-8939-27af384d4936\" (UID: \"d79c28c2-2ec8-4194-8939-27af384d4936\") "
Jul 30 03:53:23.318860 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.318826    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d79c28c2-2ec8-4194-8939-27af384d4936-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d79c28c2-2ec8-4194-8939-27af384d4936" (UID: "d79c28c2-2ec8-4194-8939-27af384d4936"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:53:23.318860 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.318840    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d79c28c2-2ec8-4194-8939-27af384d4936-kube-api-access-z8gfd" (OuterVolumeSpecName: "kube-api-access-z8gfd") pod "d79c28c2-2ec8-4194-8939-27af384d4936" (UID: "d79c28c2-2ec8-4194-8939-27af384d4936"). InnerVolumeSpecName "kube-api-access-z8gfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:53:23.417835 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.417805    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8gfd\" (UniqueName: \"kubernetes.io/projected/d79c28c2-2ec8-4194-8939-27af384d4936-kube-api-access-z8gfd\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:53:23.417835 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.417830    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d79c28c2-2ec8-4194-8939-27af384d4936-webhook-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 03:53:23.825208 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.825168    2569 generic.go:356] "Generic (PLEG): container finished" podID="d79c28c2-2ec8-4194-8939-27af384d4936" containerID="4d5f965364606bc28910b8c3196df704c65b7c53d296ded3101379ea32784bdc" exitCode=0
Jul 30 03:53:23.825658 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.825215    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm"
Jul 30 03:53:23.825658 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.825248    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm" event={"ID":"d79c28c2-2ec8-4194-8939-27af384d4936","Type":"ContainerDied","Data":"4d5f965364606bc28910b8c3196df704c65b7c53d296ded3101379ea32784bdc"}
Jul 30 03:53:23.825658 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.825301    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68d9fb7cc7-nwhgm" event={"ID":"d79c28c2-2ec8-4194-8939-27af384d4936","Type":"ContainerDied","Data":"8124c3d639a2c290616b77c7bd614824b09840d1659762d90da1ad130f94c736"}
Jul 30 03:53:23.825658 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.825330    2569 scope.go:117] "RemoveContainer" containerID="4d5f965364606bc28910b8c3196df704c65b7c53d296ded3101379ea32784bdc"
Jul 30 03:53:23.834213 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.834196    2569 scope.go:117] "RemoveContainer" containerID="4d5f965364606bc28910b8c3196df704c65b7c53d296ded3101379ea32784bdc"
Jul 30 03:53:23.834472 ip-10-0-136-104 kubenswrapper[2569]: E0730 03:53:23.834451    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d5f965364606bc28910b8c3196df704c65b7c53d296ded3101379ea32784bdc\": container with ID starting with 4d5f965364606bc28910b8c3196df704c65b7c53d296ded3101379ea32784bdc not found: ID does not exist" containerID="4d5f965364606bc28910b8c3196df704c65b7c53d296ded3101379ea32784bdc"
Jul 30 03:53:23.834553 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.834484    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d5f965364606bc28910b8c3196df704c65b7c53d296ded3101379ea32784bdc"} err="failed to get container status \"4d5f965364606bc28910b8c3196df704c65b7c53d296ded3101379ea32784bdc\": rpc error: code = NotFound desc = could not find container \"4d5f965364606bc28910b8c3196df704c65b7c53d296ded3101379ea32784bdc\": container with ID starting with 4d5f965364606bc28910b8c3196df704c65b7c53d296ded3101379ea32784bdc not found: ID does not exist"
Jul 30 03:53:23.842313 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.842289    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68d9fb7cc7-nwhgm"]
Jul 30 03:53:23.847869 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:23.847848    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68d9fb7cc7-nwhgm"]
Jul 30 03:53:25.464915 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:25.464877    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d79c28c2-2ec8-4194-8939-27af384d4936" path="/var/lib/kubelet/pods/d79c28c2-2ec8-4194-8939-27af384d4936/volumes"
Jul 30 03:53:43.437669 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:43.437626    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:43.537922 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:43.537890    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:45.045147 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.045115    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:45.084477 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.084446    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"]
Jul 30 03:53:45.084782 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.084763    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d79c28c2-2ec8-4194-8939-27af384d4936" containerName="manager"
Jul 30 03:53:45.084840 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.084783    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d79c28c2-2ec8-4194-8939-27af384d4936" containerName="manager"
Jul 30 03:53:45.084840 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.084834    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d79c28c2-2ec8-4194-8939-27af384d4936" containerName="manager"
Jul 30 03:53:45.092278 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.092258    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"
Jul 30 03:53:45.094602 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.094574    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 30 03:53:45.099070 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.099043    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"]
Jul 30 03:53:45.176350 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.176323    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4689f77d-2389-4dc5-bae4-427459c63d82-openshift-service-ca\") pod \"maas-api-86659d4467-qf4jr\" (UID: \"4689f77d-2389-4dc5-bae4-427459c63d82\") " pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"
Jul 30 03:53:45.176488 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.176373    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4689f77d-2389-4dc5-bae4-427459c63d82-maas-api-tls\") pod \"maas-api-86659d4467-qf4jr\" (UID: \"4689f77d-2389-4dc5-bae4-427459c63d82\") " pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"
Jul 30 03:53:45.176488 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.176446    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlg8z\" (UniqueName: \"kubernetes.io/projected/4689f77d-2389-4dc5-bae4-427459c63d82-kube-api-access-hlg8z\") pod \"maas-api-86659d4467-qf4jr\" (UID: \"4689f77d-2389-4dc5-bae4-427459c63d82\") " pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"
Jul 30 03:53:45.244461 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.244432    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:45.277694 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.277667    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4689f77d-2389-4dc5-bae4-427459c63d82-maas-api-tls\") pod \"maas-api-86659d4467-qf4jr\" (UID: \"4689f77d-2389-4dc5-bae4-427459c63d82\") " pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"
Jul 30 03:53:45.277779 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.277716    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlg8z\" (UniqueName: \"kubernetes.io/projected/4689f77d-2389-4dc5-bae4-427459c63d82-kube-api-access-hlg8z\") pod \"maas-api-86659d4467-qf4jr\" (UID: \"4689f77d-2389-4dc5-bae4-427459c63d82\") " pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"
Jul 30 03:53:45.277779 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.277756    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4689f77d-2389-4dc5-bae4-427459c63d82-openshift-service-ca\") pod \"maas-api-86659d4467-qf4jr\" (UID: \"4689f77d-2389-4dc5-bae4-427459c63d82\") " pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"
Jul 30 03:53:45.278437 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.278375    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4689f77d-2389-4dc5-bae4-427459c63d82-openshift-service-ca\") pod \"maas-api-86659d4467-qf4jr\" (UID: \"4689f77d-2389-4dc5-bae4-427459c63d82\") " pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"
Jul 30 03:53:45.280117 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.280097    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4689f77d-2389-4dc5-bae4-427459c63d82-maas-api-tls\") pod \"maas-api-86659d4467-qf4jr\" (UID: \"4689f77d-2389-4dc5-bae4-427459c63d82\") " pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"
Jul 30 03:53:45.285568 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.285525    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlg8z\" (UniqueName: \"kubernetes.io/projected/4689f77d-2389-4dc5-bae4-427459c63d82-kube-api-access-hlg8z\") pod \"maas-api-86659d4467-qf4jr\" (UID: \"4689f77d-2389-4dc5-bae4-427459c63d82\") " pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"
Jul 30 03:53:45.403315 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.403247    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"
Jul 30 03:53:45.517816 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.517779    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:45.548502 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.548469    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"]
Jul 30 03:53:45.551074 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:53:45.551044    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4689f77d_2389_4dc5_bae4_427459c63d82.slice/crio-25709055e1a40fd5f36063df647f692c183efad1943dacbc95be5c735906e5b6 WatchSource:0}: Error finding container 25709055e1a40fd5f36063df647f692c183efad1943dacbc95be5c735906e5b6: Status 404 returned error can't find the container with id 25709055e1a40fd5f36063df647f692c183efad1943dacbc95be5c735906e5b6
Jul 30 03:53:45.613596 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.613528    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:45.620411 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.620391    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb"]
Jul 30 03:53:45.625349 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.625333    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb"
Jul 30 03:53:45.627320 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.627302    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 30 03:53:45.634828 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.634807    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb"]
Jul 30 03:53:45.781340 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.781316    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ac76d3-11aa-4d0d-a44e-d10297c4d5cd-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4b7fb\" (UID: \"41ac76d3-11aa-4d0d-a44e-d10297c4d5cd\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb"
Jul 30 03:53:45.781465 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.781358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p7ml\" (UniqueName: \"kubernetes.io/projected/41ac76d3-11aa-4d0d-a44e-d10297c4d5cd-kube-api-access-7p7ml\") pod \"payload-pre-processing-687dbb6cf6-4b7fb\" (UID: \"41ac76d3-11aa-4d0d-a44e-d10297c4d5cd\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb"
Jul 30 03:53:45.882218 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.882186    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p7ml\" (UniqueName: \"kubernetes.io/projected/41ac76d3-11aa-4d0d-a44e-d10297c4d5cd-kube-api-access-7p7ml\") pod \"payload-pre-processing-687dbb6cf6-4b7fb\" (UID: \"41ac76d3-11aa-4d0d-a44e-d10297c4d5cd\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb"
Jul 30 03:53:45.882366 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.882264    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ac76d3-11aa-4d0d-a44e-d10297c4d5cd-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4b7fb\" (UID: \"41ac76d3-11aa-4d0d-a44e-d10297c4d5cd\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb"
Jul 30 03:53:45.882976 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.882956    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ac76d3-11aa-4d0d-a44e-d10297c4d5cd-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4b7fb\" (UID: \"41ac76d3-11aa-4d0d-a44e-d10297c4d5cd\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb"
Jul 30 03:53:45.903323 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.903302    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p7ml\" (UniqueName: \"kubernetes.io/projected/41ac76d3-11aa-4d0d-a44e-d10297c4d5cd-kube-api-access-7p7ml\") pod \"payload-pre-processing-687dbb6cf6-4b7fb\" (UID: \"41ac76d3-11aa-4d0d-a44e-d10297c4d5cd\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb"
Jul 30 03:53:45.933704 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.933675    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr" event={"ID":"4689f77d-2389-4dc5-bae4-427459c63d82","Type":"ContainerStarted","Data":"25709055e1a40fd5f36063df647f692c183efad1943dacbc95be5c735906e5b6"}
Jul 30 03:53:45.936217 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:45.936198    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb"
Jul 30 03:53:46.064869 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:46.064841    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb"]
Jul 30 03:53:46.066610 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:53:46.066583    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ac76d3_11aa_4d0d_a44e_d10297c4d5cd.slice/crio-27b9b23da7fb370e12c5d08b41e674cf99024860687680d69ef217bc85eaf10f WatchSource:0}: Error finding container 27b9b23da7fb370e12c5d08b41e674cf99024860687680d69ef217bc85eaf10f: Status 404 returned error can't find the container with id 27b9b23da7fb370e12c5d08b41e674cf99024860687680d69ef217bc85eaf10f
Jul 30 03:53:46.945742 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:46.945686    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb" event={"ID":"41ac76d3-11aa-4d0d-a44e-d10297c4d5cd","Type":"ContainerStarted","Data":"27b9b23da7fb370e12c5d08b41e674cf99024860687680d69ef217bc85eaf10f"}
Jul 30 03:53:47.951650 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:47.951617    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr" event={"ID":"4689f77d-2389-4dc5-bae4-427459c63d82","Type":"ContainerStarted","Data":"38c9c7230b8ef606e82dae3fd23297d3baa5167a082ba0b56109063f31818c52"}
Jul 30 03:53:47.952045 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:47.951704    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"
Jul 30 03:53:47.953213 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:47.953188    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb" event={"ID":"41ac76d3-11aa-4d0d-a44e-d10297c4d5cd","Type":"ContainerStarted","Data":"1eb6773d083342055d4522f698e087f742fc8af72c1e100dbc10146e57235264"}
Jul 30 03:53:47.953319 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:47.953236    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb"
Jul 30 03:53:47.967354 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:47.967315    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr" podStartSLOduration=1.398757243 podStartE2EDuration="2.967290717s" totalImagesPullingTime="1.568533474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:53:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:53:47.966516845 +0000 UTC m=+527.005227864" watchObservedRunningTime="2026-07-30 03:53:47.967290717 +0000 UTC m=+527.006001735"
Jul 30 03:53:47.980560 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:47.980483    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb" podStartSLOduration=1.2680793879999999 podStartE2EDuration="2.980464411s" totalImagesPullingTime="1.712385023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:53:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:53:47.979154904 +0000 UTC m=+527.017865922" watchObservedRunningTime="2026-07-30 03:53:47.980464411 +0000 UTC m=+527.019175528"
Jul 30 03:53:49.435325 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:49.435277    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:49.491021 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:49.490985    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:53.964261 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:53.964190    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-86659d4467-qf4jr"
Jul 30 03:53:54.940547 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:54.940496    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:55.039519 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:55.039473    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:58.965929 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:53:58.965898    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4b7fb"
Jul 30 03:54:05.571690 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.571660    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"]
Jul 30 03:54:05.576053 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.576037    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.578179 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.578154    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 03:54:05.578179 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.578171    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 03:54:05.578362 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.578223    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bwnzj\""
Jul 30 03:54:05.578738 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.578722    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 30 03:54:05.584265 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.584234    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"]
Jul 30 03:54:05.602619 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.602594    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:54:05.740940 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.740915    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/751ae2c6-4e75-46db-ba51-8e859c550f48-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.741073 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.740947    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/751ae2c6-4e75-46db-ba51-8e859c550f48-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.741073 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.740979    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/751ae2c6-4e75-46db-ba51-8e859c550f48-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.741073 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.741053    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqhw9\" (UniqueName: \"kubernetes.io/projected/751ae2c6-4e75-46db-ba51-8e859c550f48-kube-api-access-gqhw9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.741233 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.741125    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/751ae2c6-4e75-46db-ba51-8e859c550f48-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.741233 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.741162    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/751ae2c6-4e75-46db-ba51-8e859c550f48-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.841726 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.841665    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/751ae2c6-4e75-46db-ba51-8e859c550f48-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.841726 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.841706    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqhw9\" (UniqueName: \"kubernetes.io/projected/751ae2c6-4e75-46db-ba51-8e859c550f48-kube-api-access-gqhw9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.841857 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.841732    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/751ae2c6-4e75-46db-ba51-8e859c550f48-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.841857 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.841751    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/751ae2c6-4e75-46db-ba51-8e859c550f48-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.841857 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.841780    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/751ae2c6-4e75-46db-ba51-8e859c550f48-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.841857 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.841802    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/751ae2c6-4e75-46db-ba51-8e859c550f48-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.842076 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.842048    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/751ae2c6-4e75-46db-ba51-8e859c550f48-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.842163 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.842144    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/751ae2c6-4e75-46db-ba51-8e859c550f48-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.842226 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.842169    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/751ae2c6-4e75-46db-ba51-8e859c550f48-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.844146 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.844117    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/751ae2c6-4e75-46db-ba51-8e859c550f48-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.844296 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.844280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/751ae2c6-4e75-46db-ba51-8e859c550f48-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.850248 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.850225    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqhw9\" (UniqueName: \"kubernetes.io/projected/751ae2c6-4e75-46db-ba51-8e859c550f48-kube-api-access-gqhw9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4\" (UID: \"751ae2c6-4e75-46db-ba51-8e859c550f48\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:05.887881 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:05.887855    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:06.023510 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:06.023486    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"]
Jul 30 03:54:06.025563 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:54:06.025519    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod751ae2c6_4e75_46db_ba51_8e859c550f48.slice/crio-d295455fb7a138d3dda141768af36d142c8c2e6670d9832d30b499f16e9212a3 WatchSource:0}: Error finding container d295455fb7a138d3dda141768af36d142c8c2e6670d9832d30b499f16e9212a3: Status 404 returned error can't find the container with id d295455fb7a138d3dda141768af36d142c8c2e6670d9832d30b499f16e9212a3
Jul 30 03:54:06.043881 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:06.043846    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4" event={"ID":"751ae2c6-4e75-46db-ba51-8e859c550f48","Type":"ContainerStarted","Data":"d295455fb7a138d3dda141768af36d142c8c2e6670d9832d30b499f16e9212a3"}
Jul 30 03:54:08.055046 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:08.055009    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4" event={"ID":"751ae2c6-4e75-46db-ba51-8e859c550f48","Type":"ContainerStarted","Data":"b64ad61a3f938878b6057211d527276146eddd0d5d0e5196e459be74eba4e282"}
Jul 30 03:54:08.070724 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:08.070674    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4" podStartSLOduration=1.913863213 podStartE2EDuration="3.07065871s" totalImagesPullingTime="1.156795497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:54:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:54:08.069451388 +0000 UTC m=+547.108162408" watchObservedRunningTime="2026-07-30 03:54:08.07065871 +0000 UTC m=+547.109369728"
Jul 30 03:54:15.888595 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:15.888521    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:15.889000 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:15.888626    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:15.900668 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:15.900640    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:16.103383 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:16.103355    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4"
Jul 30 03:54:18.877513 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:18.877478    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:54:22.979745 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:22.979696    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:54:30.834550 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:30.834498    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:54:32.486143 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.486109    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"]
Jul 30 03:54:32.489240 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.489222    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.493941 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.493921    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 30 03:54:32.513979 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.513948    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"]
Jul 30 03:54:32.531788 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.531764    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/226f73cc-89c7-41ac-ae89-c5fbc12149e3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.531902 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.531790    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/226f73cc-89c7-41ac-ae89-c5fbc12149e3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.531902 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.531817    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/226f73cc-89c7-41ac-ae89-c5fbc12149e3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.531902 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.531835    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v8xs\" (UniqueName: \"kubernetes.io/projected/226f73cc-89c7-41ac-ae89-c5fbc12149e3-kube-api-access-4v8xs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.532102 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.531989    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/226f73cc-89c7-41ac-ae89-c5fbc12149e3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.532102 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.532035    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/226f73cc-89c7-41ac-ae89-c5fbc12149e3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.552142 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.552117    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:54:32.632995 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.632966    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/226f73cc-89c7-41ac-ae89-c5fbc12149e3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.633134 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.633006    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/226f73cc-89c7-41ac-ae89-c5fbc12149e3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.633134 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.633026    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/226f73cc-89c7-41ac-ae89-c5fbc12149e3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.633134 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.633042    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/226f73cc-89c7-41ac-ae89-c5fbc12149e3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.633134 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.633089    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/226f73cc-89c7-41ac-ae89-c5fbc12149e3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.633134 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.633110    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v8xs\" (UniqueName: \"kubernetes.io/projected/226f73cc-89c7-41ac-ae89-c5fbc12149e3-kube-api-access-4v8xs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.633387 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.633362    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/226f73cc-89c7-41ac-ae89-c5fbc12149e3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.633473 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.633454    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/226f73cc-89c7-41ac-ae89-c5fbc12149e3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.633520 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.633475    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/226f73cc-89c7-41ac-ae89-c5fbc12149e3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.635473 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.635447    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/226f73cc-89c7-41ac-ae89-c5fbc12149e3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.635742 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.635723    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/226f73cc-89c7-41ac-ae89-c5fbc12149e3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.640311 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.640291    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v8xs\" (UniqueName: \"kubernetes.io/projected/226f73cc-89c7-41ac-ae89-c5fbc12149e3-kube-api-access-4v8xs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f\" (UID: \"226f73cc-89c7-41ac-ae89-c5fbc12149e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.798892 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.798864    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:32.931196 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:32.931173    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"]
Jul 30 03:54:32.932873 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:54:32.932848    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod226f73cc_89c7_41ac_ae89_c5fbc12149e3.slice/crio-f845ac19c23766c320d2cfc23137de0149cab36a12e5686abaf84649c8ec26c7 WatchSource:0}: Error finding container f845ac19c23766c320d2cfc23137de0149cab36a12e5686abaf84649c8ec26c7: Status 404 returned error can't find the container with id f845ac19c23766c320d2cfc23137de0149cab36a12e5686abaf84649c8ec26c7
Jul 30 03:54:33.172662 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:33.172635    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f" event={"ID":"226f73cc-89c7-41ac-ae89-c5fbc12149e3","Type":"ContainerStarted","Data":"f845ac19c23766c320d2cfc23137de0149cab36a12e5686abaf84649c8ec26c7"}
Jul 30 03:54:34.179102 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:34.179068    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f" event={"ID":"226f73cc-89c7-41ac-ae89-c5fbc12149e3","Type":"ContainerStarted","Data":"fdc2695e6cd61c56d00bf52c7fdd6f8423332b912ba2321bfe67442998394cbc"}
Jul 30 03:54:34.195622 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:34.195572    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f" podStartSLOduration=2.012415371 podStartE2EDuration="2.195557185s" totalImagesPullingTime="183.141814ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:54:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:54:34.193199849 +0000 UTC m=+573.231910868" watchObservedRunningTime="2026-07-30 03:54:34.195557185 +0000 UTC m=+573.234268203"
Jul 30 03:54:42.799622 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:42.799588    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:42.800024 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:42.799655    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:42.811723 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:42.811696    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:43.232101 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:43.232015    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f"
Jul 30 03:54:47.650152 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.650123    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"]
Jul 30 03:54:47.656015 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.655998    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.658487 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.658465    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 30 03:54:47.664331 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.664310    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"]
Jul 30 03:54:47.668149 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.668123    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:54:47.744664 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.744639    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28cab731-6be8-4ec4-b837-4a662b87ef13-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.744786 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.744686    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28cab731-6be8-4ec4-b837-4a662b87ef13-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.744786 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.744737    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmb8w\" (UniqueName: \"kubernetes.io/projected/28cab731-6be8-4ec4-b837-4a662b87ef13-kube-api-access-rmb8w\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.744786 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.744757    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28cab731-6be8-4ec4-b837-4a662b87ef13-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.744890 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.744805    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28cab731-6be8-4ec4-b837-4a662b87ef13-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.744890 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.744823    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28cab731-6be8-4ec4-b837-4a662b87ef13-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.845584 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.845558    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28cab731-6be8-4ec4-b837-4a662b87ef13-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.845688 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.845602    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28cab731-6be8-4ec4-b837-4a662b87ef13-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.845688 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.845631    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmb8w\" (UniqueName: \"kubernetes.io/projected/28cab731-6be8-4ec4-b837-4a662b87ef13-kube-api-access-rmb8w\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.845688 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.845649    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28cab731-6be8-4ec4-b837-4a662b87ef13-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.845688 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.845673    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28cab731-6be8-4ec4-b837-4a662b87ef13-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.845894 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.845690    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28cab731-6be8-4ec4-b837-4a662b87ef13-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.845959 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.845936    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28cab731-6be8-4ec4-b837-4a662b87ef13-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.846076 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.846055    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28cab731-6be8-4ec4-b837-4a662b87ef13-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.846141 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.846074    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28cab731-6be8-4ec4-b837-4a662b87ef13-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.847922 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.847902    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28cab731-6be8-4ec4-b837-4a662b87ef13-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.848157 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.848141    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28cab731-6be8-4ec4-b837-4a662b87ef13-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.853001 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.852982    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmb8w\" (UniqueName: \"kubernetes.io/projected/28cab731-6be8-4ec4-b837-4a662b87ef13-kube-api-access-rmb8w\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xqlkp\" (UID: \"28cab731-6be8-4ec4-b837-4a662b87ef13\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:47.967563 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:47.967489    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:48.098774 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:48.098749    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"]
Jul 30 03:54:48.101476 ip-10-0-136-104 kubenswrapper[2569]: W0730 03:54:48.101450    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28cab731_6be8_4ec4_b837_4a662b87ef13.slice/crio-a52d288008d6eb968fa691814e1b5f2652faf2d803a75ef802b3a60b5952583d WatchSource:0}: Error finding container a52d288008d6eb968fa691814e1b5f2652faf2d803a75ef802b3a60b5952583d: Status 404 returned error can't find the container with id a52d288008d6eb968fa691814e1b5f2652faf2d803a75ef802b3a60b5952583d
Jul 30 03:54:48.246622 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:48.246525    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp" event={"ID":"28cab731-6be8-4ec4-b837-4a662b87ef13","Type":"ContainerStarted","Data":"a52d288008d6eb968fa691814e1b5f2652faf2d803a75ef802b3a60b5952583d"}
Jul 30 03:54:49.253600 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:49.253563    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp" event={"ID":"28cab731-6be8-4ec4-b837-4a662b87ef13","Type":"ContainerStarted","Data":"146479cfb7b383cc8de3869d163622866b6fae701ae77314cd9c90450f2987b7"}
Jul 30 03:54:49.270359 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:49.270310    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp" podStartSLOduration=2.077350942 podStartE2EDuration="2.270295419s" totalImagesPullingTime="192.944477ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:54:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:54:49.268621037 +0000 UTC m=+588.307332057" watchObservedRunningTime="2026-07-30 03:54:49.270295419 +0000 UTC m=+588.309006438"
Jul 30 03:54:53.922493 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:53.922453    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:54:54.023707 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:54.023680    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:54:57.968083 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:57.968045    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:57.968513 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:57.968117    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:57.980796 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:57.980772    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:54:58.313344 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:54:58.313314    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xqlkp"
Jul 30 03:55:01.405802 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:55:01.405774    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 03:55:01.406222 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:55:01.406169    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 03:55:04.570293 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:55:04.570251    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:55:59.329317 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:55:59.329283    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:56:00.675856 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:56:00.675826    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:56:13.368816 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:56:13.368775    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:56:13.416345 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:56:13.416315    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:57:19.568034 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:57:19.567987    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:58:04.943302 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:58:04.943265    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:58:06.187718 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:58:06.187681    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:58:08.884566 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:58:08.884523    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:58:15.690632 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:58:15.690601    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:58:15.793492 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:58:15.793463    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:58:40.694748 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:58:40.694664    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:59:15.402518 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:59:15.402475    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:59:15.501689 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:59:15.501654    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:59:15.614665 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:59:15.614630    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:59:21.433476 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:59:21.433436    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:59:21.495248 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:59:21.495216    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:59:21.589263 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:59:21.589229    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:59:21.725302 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:59:21.725219    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:59:21.824733 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:59:21.824682    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:59:27.436320 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:59:27.436274    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 03:59:27.497449 ip-10-0-136-104 kubenswrapper[2569]: I0730 03:59:27.497410    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:00:01.437157 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:00:01.437076    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:00:01.438457 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:00:01.438434    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:02:31.454898 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:02:31.454864    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:05:01.475881 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:05:01.475849    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:05:01.479209 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:05:01.479187    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:06:27.392242 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:06:27.392209    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:02.660643 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:08:02.660603    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:03.940074 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:08:03.940042    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:40.037548 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:09:40.037490    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:45.388754 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:09:45.388715    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:54.546814 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:09:54.546784    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:01.506132 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:10:01.506106    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:10:01.510766 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:10:01.510746    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:10:15.692227 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:10:15.692189    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:15.919469 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:10:15.919439    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:16.710623 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:12:16.710587    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:14:14.225720 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:14:14.225686    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:14:50.754558 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:14:50.754495    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:14:53.930948 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:14:53.930859    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:14:54.029262 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:14:54.029232    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:14:54.398093 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:14:54.398059    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:00.015382 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:00.015341    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:00.066172 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:00.066141    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:00.245821 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:00.245783    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:00.571806 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:00.571777    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:00.671263 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:00.671234    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:01.543114 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:01.543086    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:15:01.550352 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:01.550322    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:15:02.919803 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:02.919765    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:03.021854 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:03.021818    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:04.104772 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.104739    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"]
Jul 30 04:15:04.107976 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.107961    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:04.109949 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.109924    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5a036514-serving-cert\""
Jul 30 04:15:04.110080 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.109950    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lzvw2\""
Jul 30 04:15:04.117170 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.117142    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"]
Jul 30 04:15:04.136046 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.136023    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:04.155374 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.155341    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3a393f1-2bbc-4224-9490-445613db8961-maas-api-tls\") pod \"maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr\" (UID: \"a3a393f1-2bbc-4224-9490-445613db8961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:04.155490 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.155403    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtw4r\" (UniqueName: \"kubernetes.io/projected/a3a393f1-2bbc-4224-9490-445613db8961-kube-api-access-qtw4r\") pod \"maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr\" (UID: \"a3a393f1-2bbc-4224-9490-445613db8961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:04.155490 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.155458    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3a393f1-2bbc-4224-9490-445613db8961-openshift-service-ca\") pod \"maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr\" (UID: \"a3a393f1-2bbc-4224-9490-445613db8961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:04.256079 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.256046    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtw4r\" (UniqueName: \"kubernetes.io/projected/a3a393f1-2bbc-4224-9490-445613db8961-kube-api-access-qtw4r\") pod \"maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr\" (UID: \"a3a393f1-2bbc-4224-9490-445613db8961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:04.256249 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.256093    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3a393f1-2bbc-4224-9490-445613db8961-openshift-service-ca\") pod \"maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr\" (UID: \"a3a393f1-2bbc-4224-9490-445613db8961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:04.256249 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.256131    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3a393f1-2bbc-4224-9490-445613db8961-maas-api-tls\") pod \"maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr\" (UID: \"a3a393f1-2bbc-4224-9490-445613db8961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:04.256893 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.256857    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:04.257002 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.256899    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3a393f1-2bbc-4224-9490-445613db8961-openshift-service-ca\") pod \"maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr\" (UID: \"a3a393f1-2bbc-4224-9490-445613db8961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:04.258930 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.258908    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3a393f1-2bbc-4224-9490-445613db8961-maas-api-tls\") pod \"maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr\" (UID: \"a3a393f1-2bbc-4224-9490-445613db8961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:04.263452 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.263430    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtw4r\" (UniqueName: \"kubernetes.io/projected/a3a393f1-2bbc-4224-9490-445613db8961-kube-api-access-qtw4r\") pod \"maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr\" (UID: \"a3a393f1-2bbc-4224-9490-445613db8961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:04.420123 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.420032    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:04.552202 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.552177    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"]
Jul 30 04:15:04.553971 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:15:04.553932    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a393f1_2bbc_4224_9490_445613db8961.slice/crio-0ef135b31f5f7e8ff8ab8278f2c01c32688c37985d067b060b238a0b02dcb009 WatchSource:0}: Error finding container 0ef135b31f5f7e8ff8ab8278f2c01c32688c37985d067b060b238a0b02dcb009: Status 404 returned error can't find the container with id 0ef135b31f5f7e8ff8ab8278f2c01c32688c37985d067b060b238a0b02dcb009
Jul 30 04:15:04.555361 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.555340    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 04:15:04.666663 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.666633    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:04.770810 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.770782    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:04.808836 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.808809    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"]
Jul 30 04:15:04.813434 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.813413    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"
Jul 30 04:15:04.815364 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.815342    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5a036514-dockercfg-pwpx4\""
Jul 30 04:15:04.815459 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.815383    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5a036514\""
Jul 30 04:15:04.828302 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.828278    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"]
Jul 30 04:15:04.861581 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.861553    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80a1e825-08c4-4fc5-afc4-a9467b7807e5-plugins-config-volume\") pod \"payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb\" (UID: \"80a1e825-08c4-4fc5-afc4-a9467b7807e5\") " pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"
Jul 30 04:15:04.861678 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.861585    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q2vs\" (UniqueName: \"kubernetes.io/projected/80a1e825-08c4-4fc5-afc4-a9467b7807e5-kube-api-access-2q2vs\") pod \"payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb\" (UID: \"80a1e825-08c4-4fc5-afc4-a9467b7807e5\") " pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"
Jul 30 04:15:04.962319 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.962287    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80a1e825-08c4-4fc5-afc4-a9467b7807e5-plugins-config-volume\") pod \"payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb\" (UID: \"80a1e825-08c4-4fc5-afc4-a9467b7807e5\") " pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"
Jul 30 04:15:04.962319 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.962320    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q2vs\" (UniqueName: \"kubernetes.io/projected/80a1e825-08c4-4fc5-afc4-a9467b7807e5-kube-api-access-2q2vs\") pod \"payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb\" (UID: \"80a1e825-08c4-4fc5-afc4-a9467b7807e5\") " pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"
Jul 30 04:15:04.963055 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.963036    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80a1e825-08c4-4fc5-afc4-a9467b7807e5-plugins-config-volume\") pod \"payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb\" (UID: \"80a1e825-08c4-4fc5-afc4-a9467b7807e5\") " pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"
Jul 30 04:15:04.969894 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:04.969872    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q2vs\" (UniqueName: \"kubernetes.io/projected/80a1e825-08c4-4fc5-afc4-a9467b7807e5-kube-api-access-2q2vs\") pod \"payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb\" (UID: \"80a1e825-08c4-4fc5-afc4-a9467b7807e5\") " pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"
Jul 30 04:15:05.124304 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:05.124234    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"
Jul 30 04:15:05.153749 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:05.153712    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr" event={"ID":"a3a393f1-2bbc-4224-9490-445613db8961","Type":"ContainerStarted","Data":"0ef135b31f5f7e8ff8ab8278f2c01c32688c37985d067b060b238a0b02dcb009"}
Jul 30 04:15:05.256227 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:05.256205    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"]
Jul 30 04:15:06.160994 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:06.160953    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr" event={"ID":"a3a393f1-2bbc-4224-9490-445613db8961","Type":"ContainerStarted","Data":"1537115c07e56a6073e772edcda4bf6d7264fb7e3a5617aefc9e4300035f37f8"}
Jul 30 04:15:06.161433 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:06.161079    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:06.162523 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:06.162495    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb" event={"ID":"80a1e825-08c4-4fc5-afc4-a9467b7807e5","Type":"ContainerStarted","Data":"bf9864e2b47e6686971e529bed2512a6b292185ea0ac979b27cf081e82cc4dcd"}
Jul 30 04:15:06.162689 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:06.162563    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb" event={"ID":"80a1e825-08c4-4fc5-afc4-a9467b7807e5","Type":"ContainerStarted","Data":"7345e1566c8578ca5e44ab07b5c40d5af7f6c8a3f9cda04ac9e03e60f6b1ce47"}
Jul 30 04:15:06.162689 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:06.162592    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"
Jul 30 04:15:06.176747 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:06.176705    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr" podStartSLOduration=1.187481284 podStartE2EDuration="2.176689505s" totalImagesPullingTime="989.208221ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:15:06.175746772 +0000 UTC m=+1805.214457804" watchObservedRunningTime="2026-07-30 04:15:06.176689505 +0000 UTC m=+1805.215400523"
Jul 30 04:15:06.191981 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:06.191937    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb" podStartSLOduration=2.191921757 podStartE2EDuration="2.191921757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:15:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:15:06.190980545 +0000 UTC m=+1805.229691574" watchObservedRunningTime="2026-07-30 04:15:06.191921757 +0000 UTC m=+1805.230632778"
Jul 30 04:15:12.174156 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:12.174123    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:17.169632 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:17.169591    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"
Jul 30 04:15:32.538100 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.538066    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"]
Jul 30 04:15:32.538593 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.538333    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr" podUID="a3a393f1-2bbc-4224-9490-445613db8961" containerName="maas-api" containerID="cri-o://1537115c07e56a6073e772edcda4bf6d7264fb7e3a5617aefc9e4300035f37f8" gracePeriod=30
Jul 30 04:15:32.560237 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.560204    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:32.597317 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.597289    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"]
Jul 30 04:15:32.597591 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.597524    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb" podUID="80a1e825-08c4-4fc5-afc4-a9467b7807e5" containerName="payload-processing" containerID="cri-o://bf9864e2b47e6686971e529bed2512a6b292185ea0ac979b27cf081e82cc4dcd" gracePeriod=30
Jul 30 04:15:32.661288 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.661254    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:32.735782 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.735754    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:32.797097 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.797073    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:32.842950 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.842929    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:32.877302 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.877276    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"
Jul 30 04:15:32.885410 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.885384    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:32.900003 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.899977    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtw4r\" (UniqueName: \"kubernetes.io/projected/a3a393f1-2bbc-4224-9490-445613db8961-kube-api-access-qtw4r\") pod \"a3a393f1-2bbc-4224-9490-445613db8961\" (UID: \"a3a393f1-2bbc-4224-9490-445613db8961\") "
Jul 30 04:15:32.900122 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.900100    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3a393f1-2bbc-4224-9490-445613db8961-maas-api-tls\") pod \"a3a393f1-2bbc-4224-9490-445613db8961\" (UID: \"a3a393f1-2bbc-4224-9490-445613db8961\") "
Jul 30 04:15:32.900168 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.900134    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2q2vs\" (UniqueName: \"kubernetes.io/projected/80a1e825-08c4-4fc5-afc4-a9467b7807e5-kube-api-access-2q2vs\") pod \"80a1e825-08c4-4fc5-afc4-a9467b7807e5\" (UID: \"80a1e825-08c4-4fc5-afc4-a9467b7807e5\") "
Jul 30 04:15:32.900210 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.900170    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3a393f1-2bbc-4224-9490-445613db8961-openshift-service-ca\") pod \"a3a393f1-2bbc-4224-9490-445613db8961\" (UID: \"a3a393f1-2bbc-4224-9490-445613db8961\") "
Jul 30 04:15:32.900254 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.900206    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80a1e825-08c4-4fc5-afc4-a9467b7807e5-plugins-config-volume\") pod \"80a1e825-08c4-4fc5-afc4-a9467b7807e5\" (UID: \"80a1e825-08c4-4fc5-afc4-a9467b7807e5\") "
Jul 30 04:15:32.900633 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.900548    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3a393f1-2bbc-4224-9490-445613db8961-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a3a393f1-2bbc-4224-9490-445613db8961" (UID: "a3a393f1-2bbc-4224-9490-445613db8961"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:15:32.900771 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.900694    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80a1e825-08c4-4fc5-afc4-a9467b7807e5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "80a1e825-08c4-4fc5-afc4-a9467b7807e5" (UID: "80a1e825-08c4-4fc5-afc4-a9467b7807e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:15:32.902603 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.902568    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80a1e825-08c4-4fc5-afc4-a9467b7807e5-kube-api-access-2q2vs" (OuterVolumeSpecName: "kube-api-access-2q2vs") pod "80a1e825-08c4-4fc5-afc4-a9467b7807e5" (UID: "80a1e825-08c4-4fc5-afc4-a9467b7807e5"). InnerVolumeSpecName "kube-api-access-2q2vs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:15:32.902845 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.902826    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3a393f1-2bbc-4224-9490-445613db8961-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a3a393f1-2bbc-4224-9490-445613db8961" (UID: "a3a393f1-2bbc-4224-9490-445613db8961"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:15:32.902845 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:32.902831    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3a393f1-2bbc-4224-9490-445613db8961-kube-api-access-qtw4r" (OuterVolumeSpecName: "kube-api-access-qtw4r") pod "a3a393f1-2bbc-4224-9490-445613db8961" (UID: "a3a393f1-2bbc-4224-9490-445613db8961"). InnerVolumeSpecName "kube-api-access-qtw4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:15:33.001654 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.001624    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3a393f1-2bbc-4224-9490-445613db8961-maas-api-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:15:33.001654 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.001649    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q2vs\" (UniqueName: \"kubernetes.io/projected/80a1e825-08c4-4fc5-afc4-a9467b7807e5-kube-api-access-2q2vs\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:15:33.001654 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.001659    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3a393f1-2bbc-4224-9490-445613db8961-openshift-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:15:33.001890 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.001668    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80a1e825-08c4-4fc5-afc4-a9467b7807e5-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:15:33.001890 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.001676    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtw4r\" (UniqueName: \"kubernetes.io/projected/a3a393f1-2bbc-4224-9490-445613db8961-kube-api-access-qtw4r\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:15:33.140395 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.140317    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:33.305697 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.305661    2569 generic.go:356] "Generic (PLEG): container finished" podID="a3a393f1-2bbc-4224-9490-445613db8961" containerID="1537115c07e56a6073e772edcda4bf6d7264fb7e3a5617aefc9e4300035f37f8" exitCode=0
Jul 30 04:15:33.305878 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.305710    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"
Jul 30 04:15:33.305878 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.305734    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr" event={"ID":"a3a393f1-2bbc-4224-9490-445613db8961","Type":"ContainerDied","Data":"1537115c07e56a6073e772edcda4bf6d7264fb7e3a5617aefc9e4300035f37f8"}
Jul 30 04:15:33.305878 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.305783    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr" event={"ID":"a3a393f1-2bbc-4224-9490-445613db8961","Type":"ContainerDied","Data":"0ef135b31f5f7e8ff8ab8278f2c01c32688c37985d067b060b238a0b02dcb009"}
Jul 30 04:15:33.305878 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.305807    2569 scope.go:117] "RemoveContainer" containerID="1537115c07e56a6073e772edcda4bf6d7264fb7e3a5617aefc9e4300035f37f8"
Jul 30 04:15:33.307025 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.307002    2569 generic.go:356] "Generic (PLEG): container finished" podID="80a1e825-08c4-4fc5-afc4-a9467b7807e5" containerID="bf9864e2b47e6686971e529bed2512a6b292185ea0ac979b27cf081e82cc4dcd" exitCode=0
Jul 30 04:15:33.307159 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.307045    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"
Jul 30 04:15:33.307159 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.307091    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb" event={"ID":"80a1e825-08c4-4fc5-afc4-a9467b7807e5","Type":"ContainerDied","Data":"bf9864e2b47e6686971e529bed2512a6b292185ea0ac979b27cf081e82cc4dcd"}
Jul 30 04:15:33.307159 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.307143    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb" event={"ID":"80a1e825-08c4-4fc5-afc4-a9467b7807e5","Type":"ContainerDied","Data":"7345e1566c8578ca5e44ab07b5c40d5af7f6c8a3f9cda04ac9e03e60f6b1ce47"}
Jul 30 04:15:33.316029 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.316010    2569 scope.go:117] "RemoveContainer" containerID="1537115c07e56a6073e772edcda4bf6d7264fb7e3a5617aefc9e4300035f37f8"
Jul 30 04:15:33.316347 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:15:33.316250    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1537115c07e56a6073e772edcda4bf6d7264fb7e3a5617aefc9e4300035f37f8\": container with ID starting with 1537115c07e56a6073e772edcda4bf6d7264fb7e3a5617aefc9e4300035f37f8 not found: ID does not exist" containerID="1537115c07e56a6073e772edcda4bf6d7264fb7e3a5617aefc9e4300035f37f8"
Jul 30 04:15:33.316347 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.316275    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1537115c07e56a6073e772edcda4bf6d7264fb7e3a5617aefc9e4300035f37f8"} err="failed to get container status \"1537115c07e56a6073e772edcda4bf6d7264fb7e3a5617aefc9e4300035f37f8\": rpc error: code = NotFound desc = could not find container \"1537115c07e56a6073e772edcda4bf6d7264fb7e3a5617aefc9e4300035f37f8\": container with ID starting with 1537115c07e56a6073e772edcda4bf6d7264fb7e3a5617aefc9e4300035f37f8 not found: ID does not exist"
Jul 30 04:15:33.316347 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.316296    2569 scope.go:117] "RemoveContainer" containerID="bf9864e2b47e6686971e529bed2512a6b292185ea0ac979b27cf081e82cc4dcd"
Jul 30 04:15:33.324958 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.324941    2569 scope.go:117] "RemoveContainer" containerID="bf9864e2b47e6686971e529bed2512a6b292185ea0ac979b27cf081e82cc4dcd"
Jul 30 04:15:33.325216 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:15:33.325199    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf9864e2b47e6686971e529bed2512a6b292185ea0ac979b27cf081e82cc4dcd\": container with ID starting with bf9864e2b47e6686971e529bed2512a6b292185ea0ac979b27cf081e82cc4dcd not found: ID does not exist" containerID="bf9864e2b47e6686971e529bed2512a6b292185ea0ac979b27cf081e82cc4dcd"
Jul 30 04:15:33.325286 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.325226    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf9864e2b47e6686971e529bed2512a6b292185ea0ac979b27cf081e82cc4dcd"} err="failed to get container status \"bf9864e2b47e6686971e529bed2512a6b292185ea0ac979b27cf081e82cc4dcd\": rpc error: code = NotFound desc = could not find container \"bf9864e2b47e6686971e529bed2512a6b292185ea0ac979b27cf081e82cc4dcd\": container with ID starting with bf9864e2b47e6686971e529bed2512a6b292185ea0ac979b27cf081e82cc4dcd not found: ID does not exist"
Jul 30 04:15:33.326688 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.326670    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"]
Jul 30 04:15:33.330492 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.330468    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5a036514-56cb6b7695-fm6sr"]
Jul 30 04:15:33.340755 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.340731    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"]
Jul 30 04:15:33.343926 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.343907    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5a036514-6b5685cf74-gp7sb"]
Jul 30 04:15:33.396244 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.396162    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:33.464781 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.464749    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80a1e825-08c4-4fc5-afc4-a9467b7807e5" path="/var/lib/kubelet/pods/80a1e825-08c4-4fc5-afc4-a9467b7807e5/volumes"
Jul 30 04:15:33.465172 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.465158    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3a393f1-2bbc-4224-9490-445613db8961" path="/var/lib/kubelet/pods/a3a393f1-2bbc-4224-9490-445613db8961/volumes"
Jul 30 04:15:33.495902 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.495878    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:33.622684 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.622659    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:33.918079 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:33.918048    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:34.022198 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:34.022170    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:34.165851 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:34.165820    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:36.208742 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.208711    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:36.290801 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.290766    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"]
Jul 30 04:15:36.291181 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.291156    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80a1e825-08c4-4fc5-afc4-a9467b7807e5" containerName="payload-processing"
Jul 30 04:15:36.291181 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.291179    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="80a1e825-08c4-4fc5-afc4-a9467b7807e5" containerName="payload-processing"
Jul 30 04:15:36.291388 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.291211    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3a393f1-2bbc-4224-9490-445613db8961" containerName="maas-api"
Jul 30 04:15:36.291388 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.291221    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3a393f1-2bbc-4224-9490-445613db8961" containerName="maas-api"
Jul 30 04:15:36.291388 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.291328    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="80a1e825-08c4-4fc5-afc4-a9467b7807e5" containerName="payload-processing"
Jul 30 04:15:36.291388 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.291341    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3a393f1-2bbc-4224-9490-445613db8961" containerName="maas-api"
Jul 30 04:15:36.296140 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.296119    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:15:36.298361 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.298341    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lzvw2\""
Jul 30 04:15:36.298466 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.298349    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-974e458f-serving-cert\""
Jul 30 04:15:36.304704 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.304273    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"]
Jul 30 04:15:36.327427 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.327406    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-maas-api-tls\") pod \"maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp\" (UID: \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:15:36.327561 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.327457    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r22l\" (UniqueName: \"kubernetes.io/projected/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-kube-api-access-4r22l\") pod \"maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp\" (UID: \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:15:36.327561 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.327485    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-openshift-service-ca\") pod \"maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp\" (UID: \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:15:36.328785 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.328737    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:36.428248 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.428213    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-openshift-service-ca\") pod \"maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp\" (UID: \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:15:36.428349 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.428283    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-maas-api-tls\") pod \"maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp\" (UID: \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:15:36.428349 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.428333    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r22l\" (UniqueName: \"kubernetes.io/projected/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-kube-api-access-4r22l\") pod \"maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp\" (UID: \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:15:36.428455 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:15:36.428438    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-serving-cert: secret "maas-api-e2e-migrate-974e458f-serving-cert" not found
Jul 30 04:15:36.428525 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:15:36.428514    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-maas-api-tls podName:5955ef66-0b27-4f3e-99c5-0f1f5cebe485 nodeName:}" failed. No retries permitted until 2026-07-30 04:15:36.928494888 +0000 UTC m=+1835.967205885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-maas-api-tls") pod "maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp" (UID: "5955ef66-0b27-4f3e-99c5-0f1f5cebe485") : secret "maas-api-e2e-migrate-974e458f-serving-cert" not found
Jul 30 04:15:36.429067 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.429049    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-openshift-service-ca\") pod \"maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp\" (UID: \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:15:36.437197 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.437173    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r22l\" (UniqueName: \"kubernetes.io/projected/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-kube-api-access-4r22l\") pod \"maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp\" (UID: \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:15:36.498070 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.498008    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:36.921768 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.921738    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:36.932964 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.932942    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-maas-api-tls\") pod \"maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp\" (UID: \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:15:36.935326 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:36.935299    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-maas-api-tls\") pod \"maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp\" (UID: \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:15:37.027451 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.027413    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:37.028739 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.028719    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"]
Jul 30 04:15:37.032163 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.032145    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"
Jul 30 04:15:37.034083 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.034062    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-974e458f-dockercfg-gv7hb\""
Jul 30 04:15:37.034204 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.034099    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-974e458f\""
Jul 30 04:15:37.042243 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.042221    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"]
Jul 30 04:15:37.062347 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.062321    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"]
Jul 30 04:15:37.066107 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.066088    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"
Jul 30 04:15:37.074222 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.074200    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"]
Jul 30 04:15:37.135034 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.135007    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11796cba-74e0-4dbc-b482-24a0dc4f467f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts\" (UID: \"11796cba-74e0-4dbc-b482-24a0dc4f467f\") " pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"
Jul 30 04:15:37.135164 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.135059    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d67d3aa2-eaeb-4fd1-b039-65ab07727fb0-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl\" (UID: \"d67d3aa2-eaeb-4fd1-b039-65ab07727fb0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"
Jul 30 04:15:37.135164 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.135135    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wqt2\" (UniqueName: \"kubernetes.io/projected/d67d3aa2-eaeb-4fd1-b039-65ab07727fb0-kube-api-access-4wqt2\") pod \"payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl\" (UID: \"d67d3aa2-eaeb-4fd1-b039-65ab07727fb0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"
Jul 30 04:15:37.135242 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.135200    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdb6d\" (UniqueName: \"kubernetes.io/projected/11796cba-74e0-4dbc-b482-24a0dc4f467f-kube-api-access-wdb6d\") pod \"payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts\" (UID: \"11796cba-74e0-4dbc-b482-24a0dc4f467f\") " pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"
Jul 30 04:15:37.209063 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.209000    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:15:37.236244 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.236210    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11796cba-74e0-4dbc-b482-24a0dc4f467f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts\" (UID: \"11796cba-74e0-4dbc-b482-24a0dc4f467f\") " pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"
Jul 30 04:15:37.236389 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.236259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d67d3aa2-eaeb-4fd1-b039-65ab07727fb0-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl\" (UID: \"d67d3aa2-eaeb-4fd1-b039-65ab07727fb0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"
Jul 30 04:15:37.236389 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.236299    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wqt2\" (UniqueName: \"kubernetes.io/projected/d67d3aa2-eaeb-4fd1-b039-65ab07727fb0-kube-api-access-4wqt2\") pod \"payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl\" (UID: \"d67d3aa2-eaeb-4fd1-b039-65ab07727fb0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"
Jul 30 04:15:37.236389 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.236351    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdb6d\" (UniqueName: \"kubernetes.io/projected/11796cba-74e0-4dbc-b482-24a0dc4f467f-kube-api-access-wdb6d\") pod \"payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts\" (UID: \"11796cba-74e0-4dbc-b482-24a0dc4f467f\") " pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"
Jul 30 04:15:37.236899 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.236869    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11796cba-74e0-4dbc-b482-24a0dc4f467f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts\" (UID: \"11796cba-74e0-4dbc-b482-24a0dc4f467f\") " pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"
Jul 30 04:15:37.236986 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.236969    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d67d3aa2-eaeb-4fd1-b039-65ab07727fb0-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl\" (UID: \"d67d3aa2-eaeb-4fd1-b039-65ab07727fb0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"
Jul 30 04:15:37.244325 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.244296    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wqt2\" (UniqueName: \"kubernetes.io/projected/d67d3aa2-eaeb-4fd1-b039-65ab07727fb0-kube-api-access-4wqt2\") pod \"payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl\" (UID: \"d67d3aa2-eaeb-4fd1-b039-65ab07727fb0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"
Jul 30 04:15:37.244885 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.244859    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdb6d\" (UniqueName: \"kubernetes.io/projected/11796cba-74e0-4dbc-b482-24a0dc4f467f-kube-api-access-wdb6d\") pod \"payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts\" (UID: \"11796cba-74e0-4dbc-b482-24a0dc4f467f\") " pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"
Jul 30 04:15:37.340334 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.340311    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"]
Jul 30 04:15:37.342133 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:15:37.342105    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5955ef66_0b27_4f3e_99c5_0f1f5cebe485.slice/crio-846f2c25939e18e0c3d68ee660bbf65fd13ab0e7b1310b10e2c0998cc9dc09b6 WatchSource:0}: Error finding container 846f2c25939e18e0c3d68ee660bbf65fd13ab0e7b1310b10e2c0998cc9dc09b6: Status 404 returned error can't find the container with id 846f2c25939e18e0c3d68ee660bbf65fd13ab0e7b1310b10e2c0998cc9dc09b6
Jul 30 04:15:37.343608 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.343586    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"
Jul 30 04:15:37.375966 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.375943    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"
Jul 30 04:15:37.493167 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.493125    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"]
Jul 30 04:15:37.494591 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:15:37.494557    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd67d3aa2_eaeb_4fd1_b039_65ab07727fb0.slice/crio-275d300534a68ae37ed74aadba7190d793e0c07d1b53d3a4ddaeb0785c50cc86 WatchSource:0}: Error finding container 275d300534a68ae37ed74aadba7190d793e0c07d1b53d3a4ddaeb0785c50cc86: Status 404 returned error can't find the container with id 275d300534a68ae37ed74aadba7190d793e0c07d1b53d3a4ddaeb0785c50cc86
Jul 30 04:15:37.524781 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:37.524759    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"]
Jul 30 04:15:37.533742 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:15:37.533712    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11796cba_74e0_4dbc_b482_24a0dc4f467f.slice/crio-ab94fd957d8feaf8915280e36baa9939ee8f00f841f551b50c0bbe9056ba4382 WatchSource:0}: Error finding container ab94fd957d8feaf8915280e36baa9939ee8f00f841f551b50c0bbe9056ba4382: Status 404 returned error can't find the container with id ab94fd957d8feaf8915280e36baa9939ee8f00f841f551b50c0bbe9056ba4382
Jul 30 04:15:38.338517 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:38.338475    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts" event={"ID":"11796cba-74e0-4dbc-b482-24a0dc4f467f","Type":"ContainerStarted","Data":"a61ecb8385bce963005f436022861a4edf3b2f7adfbdf8ba5d5ce6864977efc0"}
Jul 30 04:15:38.339020 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:38.338569    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts" event={"ID":"11796cba-74e0-4dbc-b482-24a0dc4f467f","Type":"ContainerStarted","Data":"ab94fd957d8feaf8915280e36baa9939ee8f00f841f551b50c0bbe9056ba4382"}
Jul 30 04:15:38.339020 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:38.338603    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"
Jul 30 04:15:38.339908 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:38.339878    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl" event={"ID":"d67d3aa2-eaeb-4fd1-b039-65ab07727fb0","Type":"ContainerStarted","Data":"425a1f96b90d72f9fd1bba2ec179c814a97b8b8e1c071fec2cd029546e40b69e"}
Jul 30 04:15:38.340042 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:38.339925    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl" event={"ID":"d67d3aa2-eaeb-4fd1-b039-65ab07727fb0","Type":"ContainerStarted","Data":"275d300534a68ae37ed74aadba7190d793e0c07d1b53d3a4ddaeb0785c50cc86"}
Jul 30 04:15:38.340042 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:38.339970    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"
Jul 30 04:15:38.341409 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:38.341387    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp" event={"ID":"5955ef66-0b27-4f3e-99c5-0f1f5cebe485","Type":"ContainerStarted","Data":"ea1522a350fe2f4c225548c502d36ecd22b1e33bac6754504a55fc14cae0e901"}
Jul 30 04:15:38.341500 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:38.341426    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp" event={"ID":"5955ef66-0b27-4f3e-99c5-0f1f5cebe485","Type":"ContainerStarted","Data":"846f2c25939e18e0c3d68ee660bbf65fd13ab0e7b1310b10e2c0998cc9dc09b6"}
Jul 30 04:15:38.341500 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:38.341463    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:15:38.354286 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:38.354247    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts" podStartSLOduration=1.354235201 podStartE2EDuration="1.354235201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:15:38.353159682 +0000 UTC m=+1837.391870700" watchObservedRunningTime="2026-07-30 04:15:38.354235201 +0000 UTC m=+1837.392946219"
Jul 30 04:15:38.367591 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:38.367524    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl" podStartSLOduration=1.3675088579999999 podStartE2EDuration="1.367508858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:15:38.366863894 +0000 UTC m=+1837.405574917" watchObservedRunningTime="2026-07-30 04:15:38.367508858 +0000 UTC m=+1837.406219879"
Jul 30 04:15:38.382291 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:38.382252    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp" podStartSLOduration=1.969374734 podStartE2EDuration="2.382240052s" totalImagesPullingTime="412.865318ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:15:38.380951 +0000 UTC m=+1837.419662019" watchObservedRunningTime="2026-07-30 04:15:38.382240052 +0000 UTC m=+1837.420951070"
Jul 30 04:15:44.352806 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:44.352763    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:15:49.348724 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:49.348693    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"
Jul 30 04:15:49.349208 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:15:49.348768    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"
Jul 30 04:16:06.931523 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:06.931488    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"]
Jul 30 04:16:06.932051 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:06.931761    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp" podUID="5955ef66-0b27-4f3e-99c5-0f1f5cebe485" containerName="maas-api" containerID="cri-o://ea1522a350fe2f4c225548c502d36ecd22b1e33bac6754504a55fc14cae0e901" gracePeriod=30
Jul 30 04:16:06.960685 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:06.960655    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:06.982743 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:06.982720    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"]
Jul 30 04:16:06.983151 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:06.983120    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts" podUID="11796cba-74e0-4dbc-b482-24a0dc4f467f" containerName="payload-processing" containerID="cri-o://a61ecb8385bce963005f436022861a4edf3b2f7adfbdf8ba5d5ce6864977efc0" gracePeriod=30
Jul 30 04:16:06.989408 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:06.989384    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"]
Jul 30 04:16:06.989595 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:06.989571    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl" podUID="d67d3aa2-eaeb-4fd1-b039-65ab07727fb0" containerName="payload-pre-processing" containerID="cri-o://425a1f96b90d72f9fd1bba2ec179c814a97b8b8e1c071fec2cd029546e40b69e" gracePeriod=30
Jul 30 04:16:07.060937 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.060910    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:07.129357 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.129271    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:07.269025 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.268998    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"
Jul 30 04:16:07.282248 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.282227    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"
Jul 30 04:16:07.289805 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.289783    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:16:07.293408 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.293386    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:07.376968 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.376936    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdb6d\" (UniqueName: \"kubernetes.io/projected/11796cba-74e0-4dbc-b482-24a0dc4f467f-kube-api-access-wdb6d\") pod \"11796cba-74e0-4dbc-b482-24a0dc4f467f\" (UID: \"11796cba-74e0-4dbc-b482-24a0dc4f467f\") "
Jul 30 04:16:07.377119 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.376983    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r22l\" (UniqueName: \"kubernetes.io/projected/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-kube-api-access-4r22l\") pod \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\" (UID: \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\") "
Jul 30 04:16:07.377119 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.377003    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wqt2\" (UniqueName: \"kubernetes.io/projected/d67d3aa2-eaeb-4fd1-b039-65ab07727fb0-kube-api-access-4wqt2\") pod \"d67d3aa2-eaeb-4fd1-b039-65ab07727fb0\" (UID: \"d67d3aa2-eaeb-4fd1-b039-65ab07727fb0\") "
Jul 30 04:16:07.377119 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.377045    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11796cba-74e0-4dbc-b482-24a0dc4f467f-plugins-config-volume\") pod \"11796cba-74e0-4dbc-b482-24a0dc4f467f\" (UID: \"11796cba-74e0-4dbc-b482-24a0dc4f467f\") "
Jul 30 04:16:07.377119 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.377070    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d67d3aa2-eaeb-4fd1-b039-65ab07727fb0-plugins-config-volume\") pod \"d67d3aa2-eaeb-4fd1-b039-65ab07727fb0\" (UID: \"d67d3aa2-eaeb-4fd1-b039-65ab07727fb0\") "
Jul 30 04:16:07.377119 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.377113    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-openshift-service-ca\") pod \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\" (UID: \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\") "
Jul 30 04:16:07.377354 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.377157    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-maas-api-tls\") pod \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\" (UID: \"5955ef66-0b27-4f3e-99c5-0f1f5cebe485\") "
Jul 30 04:16:07.377522 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.377473    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11796cba-74e0-4dbc-b482-24a0dc4f467f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "11796cba-74e0-4dbc-b482-24a0dc4f467f" (UID: "11796cba-74e0-4dbc-b482-24a0dc4f467f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:16:07.377842 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.377557    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d67d3aa2-eaeb-4fd1-b039-65ab07727fb0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d67d3aa2-eaeb-4fd1-b039-65ab07727fb0" (UID: "d67d3aa2-eaeb-4fd1-b039-65ab07727fb0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:16:07.377842 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.377581    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5955ef66-0b27-4f3e-99c5-0f1f5cebe485" (UID: "5955ef66-0b27-4f3e-99c5-0f1f5cebe485"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:16:07.379323 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.379301    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-kube-api-access-4r22l" (OuterVolumeSpecName: "kube-api-access-4r22l") pod "5955ef66-0b27-4f3e-99c5-0f1f5cebe485" (UID: "5955ef66-0b27-4f3e-99c5-0f1f5cebe485"). InnerVolumeSpecName "kube-api-access-4r22l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:16:07.379414 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.379326    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d67d3aa2-eaeb-4fd1-b039-65ab07727fb0-kube-api-access-4wqt2" (OuterVolumeSpecName: "kube-api-access-4wqt2") pod "d67d3aa2-eaeb-4fd1-b039-65ab07727fb0" (UID: "d67d3aa2-eaeb-4fd1-b039-65ab07727fb0"). InnerVolumeSpecName "kube-api-access-4wqt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:16:07.379414 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.379324    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11796cba-74e0-4dbc-b482-24a0dc4f467f-kube-api-access-wdb6d" (OuterVolumeSpecName: "kube-api-access-wdb6d") pod "11796cba-74e0-4dbc-b482-24a0dc4f467f" (UID: "11796cba-74e0-4dbc-b482-24a0dc4f467f"). InnerVolumeSpecName "kube-api-access-wdb6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:16:07.379414 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.379367    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5955ef66-0b27-4f3e-99c5-0f1f5cebe485" (UID: "5955ef66-0b27-4f3e-99c5-0f1f5cebe485"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:16:07.477860 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.477842    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11796cba-74e0-4dbc-b482-24a0dc4f467f-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:16:07.477935 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.477861    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d67d3aa2-eaeb-4fd1-b039-65ab07727fb0-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:16:07.477935 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.477872    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-openshift-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:16:07.477935 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.477880    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-maas-api-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:16:07.477935 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.477889    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdb6d\" (UniqueName: \"kubernetes.io/projected/11796cba-74e0-4dbc-b482-24a0dc4f467f-kube-api-access-wdb6d\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:16:07.477935 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.477916    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r22l\" (UniqueName: \"kubernetes.io/projected/5955ef66-0b27-4f3e-99c5-0f1f5cebe485-kube-api-access-4r22l\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:16:07.477935 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.477925    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wqt2\" (UniqueName: \"kubernetes.io/projected/d67d3aa2-eaeb-4fd1-b039-65ab07727fb0-kube-api-access-4wqt2\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:16:07.493409 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.493386    2569 generic.go:356] "Generic (PLEG): container finished" podID="11796cba-74e0-4dbc-b482-24a0dc4f467f" containerID="a61ecb8385bce963005f436022861a4edf3b2f7adfbdf8ba5d5ce6864977efc0" exitCode=0
Jul 30 04:16:07.493517 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.493427    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"
Jul 30 04:16:07.493517 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.493471    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts" event={"ID":"11796cba-74e0-4dbc-b482-24a0dc4f467f","Type":"ContainerDied","Data":"a61ecb8385bce963005f436022861a4edf3b2f7adfbdf8ba5d5ce6864977efc0"}
Jul 30 04:16:07.493662 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.493521    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts" event={"ID":"11796cba-74e0-4dbc-b482-24a0dc4f467f","Type":"ContainerDied","Data":"ab94fd957d8feaf8915280e36baa9939ee8f00f841f551b50c0bbe9056ba4382"}
Jul 30 04:16:07.493662 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.493567    2569 scope.go:117] "RemoveContainer" containerID="a61ecb8385bce963005f436022861a4edf3b2f7adfbdf8ba5d5ce6864977efc0"
Jul 30 04:16:07.494895 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.494873    2569 generic.go:356] "Generic (PLEG): container finished" podID="d67d3aa2-eaeb-4fd1-b039-65ab07727fb0" containerID="425a1f96b90d72f9fd1bba2ec179c814a97b8b8e1c071fec2cd029546e40b69e" exitCode=0
Jul 30 04:16:07.494998 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.494909    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"
Jul 30 04:16:07.494998 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.494933    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl" event={"ID":"d67d3aa2-eaeb-4fd1-b039-65ab07727fb0","Type":"ContainerDied","Data":"425a1f96b90d72f9fd1bba2ec179c814a97b8b8e1c071fec2cd029546e40b69e"}
Jul 30 04:16:07.494998 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.494987    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl" event={"ID":"d67d3aa2-eaeb-4fd1-b039-65ab07727fb0","Type":"ContainerDied","Data":"275d300534a68ae37ed74aadba7190d793e0c07d1b53d3a4ddaeb0785c50cc86"}
Jul 30 04:16:07.496576 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.496523    2569 generic.go:356] "Generic (PLEG): container finished" podID="5955ef66-0b27-4f3e-99c5-0f1f5cebe485" containerID="ea1522a350fe2f4c225548c502d36ecd22b1e33bac6754504a55fc14cae0e901" exitCode=0
Jul 30 04:16:07.496576 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.496568    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp" event={"ID":"5955ef66-0b27-4f3e-99c5-0f1f5cebe485","Type":"ContainerDied","Data":"ea1522a350fe2f4c225548c502d36ecd22b1e33bac6754504a55fc14cae0e901"}
Jul 30 04:16:07.496735 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.496617    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp" event={"ID":"5955ef66-0b27-4f3e-99c5-0f1f5cebe485","Type":"ContainerDied","Data":"846f2c25939e18e0c3d68ee660bbf65fd13ab0e7b1310b10e2c0998cc9dc09b6"}
Jul 30 04:16:07.496826 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.496803    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"
Jul 30 04:16:07.503255 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.503237    2569 scope.go:117] "RemoveContainer" containerID="a61ecb8385bce963005f436022861a4edf3b2f7adfbdf8ba5d5ce6864977efc0"
Jul 30 04:16:07.503506 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:16:07.503484    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a61ecb8385bce963005f436022861a4edf3b2f7adfbdf8ba5d5ce6864977efc0\": container with ID starting with a61ecb8385bce963005f436022861a4edf3b2f7adfbdf8ba5d5ce6864977efc0 not found: ID does not exist" containerID="a61ecb8385bce963005f436022861a4edf3b2f7adfbdf8ba5d5ce6864977efc0"
Jul 30 04:16:07.503590 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.503514    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a61ecb8385bce963005f436022861a4edf3b2f7adfbdf8ba5d5ce6864977efc0"} err="failed to get container status \"a61ecb8385bce963005f436022861a4edf3b2f7adfbdf8ba5d5ce6864977efc0\": rpc error: code = NotFound desc = could not find container \"a61ecb8385bce963005f436022861a4edf3b2f7adfbdf8ba5d5ce6864977efc0\": container with ID starting with a61ecb8385bce963005f436022861a4edf3b2f7adfbdf8ba5d5ce6864977efc0 not found: ID does not exist"
Jul 30 04:16:07.503590 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.503546    2569 scope.go:117] "RemoveContainer" containerID="425a1f96b90d72f9fd1bba2ec179c814a97b8b8e1c071fec2cd029546e40b69e"
Jul 30 04:16:07.511116 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.511072    2569 scope.go:117] "RemoveContainer" containerID="425a1f96b90d72f9fd1bba2ec179c814a97b8b8e1c071fec2cd029546e40b69e"
Jul 30 04:16:07.511408 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:16:07.511384    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"425a1f96b90d72f9fd1bba2ec179c814a97b8b8e1c071fec2cd029546e40b69e\": container with ID starting with 425a1f96b90d72f9fd1bba2ec179c814a97b8b8e1c071fec2cd029546e40b69e not found: ID does not exist" containerID="425a1f96b90d72f9fd1bba2ec179c814a97b8b8e1c071fec2cd029546e40b69e"
Jul 30 04:16:07.511560 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.511421    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"425a1f96b90d72f9fd1bba2ec179c814a97b8b8e1c071fec2cd029546e40b69e"} err="failed to get container status \"425a1f96b90d72f9fd1bba2ec179c814a97b8b8e1c071fec2cd029546e40b69e\": rpc error: code = NotFound desc = could not find container \"425a1f96b90d72f9fd1bba2ec179c814a97b8b8e1c071fec2cd029546e40b69e\": container with ID starting with 425a1f96b90d72f9fd1bba2ec179c814a97b8b8e1c071fec2cd029546e40b69e not found: ID does not exist"
Jul 30 04:16:07.511560 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.511446    2569 scope.go:117] "RemoveContainer" containerID="ea1522a350fe2f4c225548c502d36ecd22b1e33bac6754504a55fc14cae0e901"
Jul 30 04:16:07.513973 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.513955    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"]
Jul 30 04:16:07.519295 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.519275    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-974e458f-78467584d-pqmnl"]
Jul 30 04:16:07.519588 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.519575    2569 scope.go:117] "RemoveContainer" containerID="ea1522a350fe2f4c225548c502d36ecd22b1e33bac6754504a55fc14cae0e901"
Jul 30 04:16:07.519841 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:16:07.519821    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea1522a350fe2f4c225548c502d36ecd22b1e33bac6754504a55fc14cae0e901\": container with ID starting with ea1522a350fe2f4c225548c502d36ecd22b1e33bac6754504a55fc14cae0e901 not found: ID does not exist" containerID="ea1522a350fe2f4c225548c502d36ecd22b1e33bac6754504a55fc14cae0e901"
Jul 30 04:16:07.519933 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.519849    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea1522a350fe2f4c225548c502d36ecd22b1e33bac6754504a55fc14cae0e901"} err="failed to get container status \"ea1522a350fe2f4c225548c502d36ecd22b1e33bac6754504a55fc14cae0e901\": rpc error: code = NotFound desc = could not find container \"ea1522a350fe2f4c225548c502d36ecd22b1e33bac6754504a55fc14cae0e901\": container with ID starting with ea1522a350fe2f4c225548c502d36ecd22b1e33bac6754504a55fc14cae0e901 not found: ID does not exist"
Jul 30 04:16:07.521112 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.521092    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:07.528465 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.528444    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"]
Jul 30 04:16:07.532222 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.532200    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-974e458f-6c488b99f7-s5jbp"]
Jul 30 04:16:07.541410 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.541389    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"]
Jul 30 04:16:07.545710 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.545690    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-974e458f-5fb46cf4b9-gjkts"]
Jul 30 04:16:07.738676 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:07.738599    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:08.052235 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:08.052202    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:08.538096 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:08.538067    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:09.466784 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:09.466752    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11796cba-74e0-4dbc-b482-24a0dc4f467f" path="/var/lib/kubelet/pods/11796cba-74e0-4dbc-b482-24a0dc4f467f/volumes"
Jul 30 04:16:09.467152 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:09.467115    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5955ef66-0b27-4f3e-99c5-0f1f5cebe485" path="/var/lib/kubelet/pods/5955ef66-0b27-4f3e-99c5-0f1f5cebe485/volumes"
Jul 30 04:16:09.467413 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:09.467401    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d67d3aa2-eaeb-4fd1-b039-65ab07727fb0" path="/var/lib/kubelet/pods/d67d3aa2-eaeb-4fd1-b039-65ab07727fb0/volumes"
Jul 30 04:16:10.265376 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:10.265347    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:10.365801 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:10.365766    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:11.360491 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.360454    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"]
Jul 30 04:16:11.360987 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.360970    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5955ef66-0b27-4f3e-99c5-0f1f5cebe485" containerName="maas-api"
Jul 30 04:16:11.361044 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.360991    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5955ef66-0b27-4f3e-99c5-0f1f5cebe485" containerName="maas-api"
Jul 30 04:16:11.361044 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.361021    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d67d3aa2-eaeb-4fd1-b039-65ab07727fb0" containerName="payload-pre-processing"
Jul 30 04:16:11.361044 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.361030    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d67d3aa2-eaeb-4fd1-b039-65ab07727fb0" containerName="payload-pre-processing"
Jul 30 04:16:11.361139 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.361049    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11796cba-74e0-4dbc-b482-24a0dc4f467f" containerName="payload-processing"
Jul 30 04:16:11.361139 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.361059    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="11796cba-74e0-4dbc-b482-24a0dc4f467f" containerName="payload-processing"
Jul 30 04:16:11.361139 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.361132    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="11796cba-74e0-4dbc-b482-24a0dc4f467f" containerName="payload-processing"
Jul 30 04:16:11.361228 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.361142    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5955ef66-0b27-4f3e-99c5-0f1f5cebe485" containerName="maas-api"
Jul 30 04:16:11.361228 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.361152    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d67d3aa2-eaeb-4fd1-b039-65ab07727fb0" containerName="payload-pre-processing"
Jul 30 04:16:11.366059 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.366033    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:11.368420 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.368391    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b51d8f16-serving-cert\""
Jul 30 04:16:11.368420 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.368411    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lzvw2\""
Jul 30 04:16:11.373155 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.373131    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"]
Jul 30 04:16:11.394030 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.393976    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:11.454522 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.454494    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:11.510522 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.510486    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2qq7\" (UniqueName: \"kubernetes.io/projected/b767e2cf-6d19-4785-9f0e-a917b06e4c45-kube-api-access-b2qq7\") pod \"maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m\" (UID: \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:11.510710 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.510586    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b767e2cf-6d19-4785-9f0e-a917b06e4c45-openshift-service-ca\") pod \"maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m\" (UID: \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:11.510710 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.510662    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b767e2cf-6d19-4785-9f0e-a917b06e4c45-maas-api-tls\") pod \"maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m\" (UID: \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:11.611261 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.611172    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b767e2cf-6d19-4785-9f0e-a917b06e4c45-openshift-service-ca\") pod \"maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m\" (UID: \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:11.611261 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.611225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b767e2cf-6d19-4785-9f0e-a917b06e4c45-maas-api-tls\") pod \"maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m\" (UID: \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:11.611485 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.611296    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2qq7\" (UniqueName: \"kubernetes.io/projected/b767e2cf-6d19-4785-9f0e-a917b06e4c45-kube-api-access-b2qq7\") pod \"maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m\" (UID: \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:11.611895 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.611871    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b767e2cf-6d19-4785-9f0e-a917b06e4c45-openshift-service-ca\") pod \"maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m\" (UID: \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:11.614027 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.613997    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b767e2cf-6d19-4785-9f0e-a917b06e4c45-maas-api-tls\") pod \"maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m\" (UID: \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:11.618813 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.618796    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2qq7\" (UniqueName: \"kubernetes.io/projected/b767e2cf-6d19-4785-9f0e-a917b06e4c45-kube-api-access-b2qq7\") pod \"maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m\" (UID: \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:11.679833 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.679805    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:11.835513 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:11.835489    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"]
Jul 30 04:16:11.837110 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:16:11.837079    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb767e2cf_6d19_4785_9f0e_a917b06e4c45.slice/crio-9db3c4cb254fac89d14a6b4b81030f1af0250530ed973f41a0baf46d94232960 WatchSource:0}: Error finding container 9db3c4cb254fac89d14a6b4b81030f1af0250530ed973f41a0baf46d94232960: Status 404 returned error can't find the container with id 9db3c4cb254fac89d14a6b4b81030f1af0250530ed973f41a0baf46d94232960
Jul 30 04:16:12.010108 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.010031    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:12.105272 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.105212    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:12.124824 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.124797    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"]
Jul 30 04:16:12.129144 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.129124    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"
Jul 30 04:16:12.131172 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.131154    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b51d8f16-dockercfg-d9qln\""
Jul 30 04:16:12.131260 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.131211    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b51d8f16\""
Jul 30 04:16:12.138930 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.138904    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"]
Jul 30 04:16:12.217611 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.217578    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/568901ab-01b2-4202-8c75-6894e157686e-plugins-config-volume\") pod \"payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt\" (UID: \"568901ab-01b2-4202-8c75-6894e157686e\") " pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"
Jul 30 04:16:12.217761 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.217635    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl2b5\" (UniqueName: \"kubernetes.io/projected/568901ab-01b2-4202-8c75-6894e157686e-kube-api-access-kl2b5\") pod \"payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt\" (UID: \"568901ab-01b2-4202-8c75-6894e157686e\") " pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"
Jul 30 04:16:12.318431 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.318399    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/568901ab-01b2-4202-8c75-6894e157686e-plugins-config-volume\") pod \"payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt\" (UID: \"568901ab-01b2-4202-8c75-6894e157686e\") " pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"
Jul 30 04:16:12.318610 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.318445    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl2b5\" (UniqueName: \"kubernetes.io/projected/568901ab-01b2-4202-8c75-6894e157686e-kube-api-access-kl2b5\") pod \"payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt\" (UID: \"568901ab-01b2-4202-8c75-6894e157686e\") " pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"
Jul 30 04:16:12.319290 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.319257    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/568901ab-01b2-4202-8c75-6894e157686e-plugins-config-volume\") pod \"payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt\" (UID: \"568901ab-01b2-4202-8c75-6894e157686e\") " pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"
Jul 30 04:16:12.327221 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.327198    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl2b5\" (UniqueName: \"kubernetes.io/projected/568901ab-01b2-4202-8c75-6894e157686e-kube-api-access-kl2b5\") pod \"payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt\" (UID: \"568901ab-01b2-4202-8c75-6894e157686e\") " pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"
Jul 30 04:16:12.441126 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.441084    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"
Jul 30 04:16:12.533647 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.533613    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m" event={"ID":"b767e2cf-6d19-4785-9f0e-a917b06e4c45","Type":"ContainerStarted","Data":"80d111625636d69df40f06a91f5313a4b80a4b5ae69f92c90cfcecbb6c7808de"}
Jul 30 04:16:12.533788 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.533713    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m" event={"ID":"b767e2cf-6d19-4785-9f0e-a917b06e4c45","Type":"ContainerStarted","Data":"9db3c4cb254fac89d14a6b4b81030f1af0250530ed973f41a0baf46d94232960"}
Jul 30 04:16:12.533788 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.533779    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:12.550746 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.550694    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m" podStartSLOduration=1.060083521 podStartE2EDuration="1.550678469s" totalImagesPullingTime="490.594948ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:16:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:16:12.548705218 +0000 UTC m=+1871.587416237" watchObservedRunningTime="2026-07-30 04:16:12.550678469 +0000 UTC m=+1871.589389488"
Jul 30 04:16:12.581849 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:12.581785    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"]
Jul 30 04:16:12.584011 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:16:12.583980    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod568901ab_01b2_4202_8c75_6894e157686e.slice/crio-9b58ecc071079bd395b7f131cedea468097378c733d3349585cad394d90d31f6 WatchSource:0}: Error finding container 9b58ecc071079bd395b7f131cedea468097378c733d3349585cad394d90d31f6: Status 404 returned error can't find the container with id 9b58ecc071079bd395b7f131cedea468097378c733d3349585cad394d90d31f6
Jul 30 04:16:13.539715 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:13.539678    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt" event={"ID":"568901ab-01b2-4202-8c75-6894e157686e","Type":"ContainerStarted","Data":"b4e305d9cdada99faaafa39a311fbedec0f7aaf54bd6fecddfc69b01446866cb"}
Jul 30 04:16:13.540129 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:13.539741    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt" event={"ID":"568901ab-01b2-4202-8c75-6894e157686e","Type":"ContainerStarted","Data":"9b58ecc071079bd395b7f131cedea468097378c733d3349585cad394d90d31f6"}
Jul 30 04:16:13.540129 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:13.539833    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"
Jul 30 04:16:13.554795 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:13.554739    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt" podStartSLOduration=1.5547243339999999 podStartE2EDuration="1.554724334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:16:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:16:13.552870485 +0000 UTC m=+1872.591581502" watchObservedRunningTime="2026-07-30 04:16:13.554724334 +0000 UTC m=+1872.593435353"
Jul 30 04:16:18.545333 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:18.545306    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:24.546956 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:24.546879    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"
Jul 30 04:16:41.933295 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:41.933262    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"]
Jul 30 04:16:41.951630 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:41.933579    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m" podUID="b767e2cf-6d19-4785-9f0e-a917b06e4c45" containerName="maas-api" containerID="cri-o://80d111625636d69df40f06a91f5313a4b80a4b5ae69f92c90cfcecbb6c7808de" gracePeriod=30
Jul 30 04:16:41.956900 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:41.956677    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:41.975568 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:41.975525    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"]
Jul 30 04:16:41.975809 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:41.975785    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt" podUID="568901ab-01b2-4202-8c75-6894e157686e" containerName="payload-processing" containerID="cri-o://b4e305d9cdada99faaafa39a311fbedec0f7aaf54bd6fecddfc69b01446866cb" gracePeriod=30
Jul 30 04:16:42.054477 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.054449    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:42.147498 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.147273    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:42.217780 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.217751    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:42.222381 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.222361    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:42.248580 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.248555    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"
Jul 30 04:16:42.262951 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.262926    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b767e2cf-6d19-4785-9f0e-a917b06e4c45-maas-api-tls\") pod \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\" (UID: \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\") "
Jul 30 04:16:42.263046 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.262968    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2qq7\" (UniqueName: \"kubernetes.io/projected/b767e2cf-6d19-4785-9f0e-a917b06e4c45-kube-api-access-b2qq7\") pod \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\" (UID: \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\") "
Jul 30 04:16:42.263046 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.263024    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b767e2cf-6d19-4785-9f0e-a917b06e4c45-openshift-service-ca\") pod \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\" (UID: \"b767e2cf-6d19-4785-9f0e-a917b06e4c45\") "
Jul 30 04:16:42.263901 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.263829    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b767e2cf-6d19-4785-9f0e-a917b06e4c45-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b767e2cf-6d19-4785-9f0e-a917b06e4c45" (UID: "b767e2cf-6d19-4785-9f0e-a917b06e4c45"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:16:42.265574 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.265411    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b767e2cf-6d19-4785-9f0e-a917b06e4c45-kube-api-access-b2qq7" (OuterVolumeSpecName: "kube-api-access-b2qq7") pod "b767e2cf-6d19-4785-9f0e-a917b06e4c45" (UID: "b767e2cf-6d19-4785-9f0e-a917b06e4c45"). InnerVolumeSpecName "kube-api-access-b2qq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:16:42.265892 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.265868    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b767e2cf-6d19-4785-9f0e-a917b06e4c45-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b767e2cf-6d19-4785-9f0e-a917b06e4c45" (UID: "b767e2cf-6d19-4785-9f0e-a917b06e4c45"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:16:42.364100 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.364061    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kl2b5\" (UniqueName: \"kubernetes.io/projected/568901ab-01b2-4202-8c75-6894e157686e-kube-api-access-kl2b5\") pod \"568901ab-01b2-4202-8c75-6894e157686e\" (UID: \"568901ab-01b2-4202-8c75-6894e157686e\") "
Jul 30 04:16:42.364100 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.364106    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/568901ab-01b2-4202-8c75-6894e157686e-plugins-config-volume\") pod \"568901ab-01b2-4202-8c75-6894e157686e\" (UID: \"568901ab-01b2-4202-8c75-6894e157686e\") "
Jul 30 04:16:42.364346 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.364330    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b767e2cf-6d19-4785-9f0e-a917b06e4c45-maas-api-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:16:42.364385 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.364354    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2qq7\" (UniqueName: \"kubernetes.io/projected/b767e2cf-6d19-4785-9f0e-a917b06e4c45-kube-api-access-b2qq7\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:16:42.364385 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.364368    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b767e2cf-6d19-4785-9f0e-a917b06e4c45-openshift-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:16:42.364471 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.364449    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/568901ab-01b2-4202-8c75-6894e157686e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "568901ab-01b2-4202-8c75-6894e157686e" (UID: "568901ab-01b2-4202-8c75-6894e157686e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:16:42.366170 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.366152    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/568901ab-01b2-4202-8c75-6894e157686e-kube-api-access-kl2b5" (OuterVolumeSpecName: "kube-api-access-kl2b5") pod "568901ab-01b2-4202-8c75-6894e157686e" (UID: "568901ab-01b2-4202-8c75-6894e157686e"). InnerVolumeSpecName "kube-api-access-kl2b5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:16:42.465464 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.465402    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl2b5\" (UniqueName: \"kubernetes.io/projected/568901ab-01b2-4202-8c75-6894e157686e-kube-api-access-kl2b5\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:16:42.465464 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.465423    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/568901ab-01b2-4202-8c75-6894e157686e-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:16:42.688710 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.688678    2569 generic.go:356] "Generic (PLEG): container finished" podID="b767e2cf-6d19-4785-9f0e-a917b06e4c45" containerID="80d111625636d69df40f06a91f5313a4b80a4b5ae69f92c90cfcecbb6c7808de" exitCode=0
Jul 30 04:16:42.688861 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.688719    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"
Jul 30 04:16:42.688861 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.688765    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m" event={"ID":"b767e2cf-6d19-4785-9f0e-a917b06e4c45","Type":"ContainerDied","Data":"80d111625636d69df40f06a91f5313a4b80a4b5ae69f92c90cfcecbb6c7808de"}
Jul 30 04:16:42.688861 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.688803    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m" event={"ID":"b767e2cf-6d19-4785-9f0e-a917b06e4c45","Type":"ContainerDied","Data":"9db3c4cb254fac89d14a6b4b81030f1af0250530ed973f41a0baf46d94232960"}
Jul 30 04:16:42.688861 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.688831    2569 scope.go:117] "RemoveContainer" containerID="80d111625636d69df40f06a91f5313a4b80a4b5ae69f92c90cfcecbb6c7808de"
Jul 30 04:16:42.690159 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.690132    2569 generic.go:356] "Generic (PLEG): container finished" podID="568901ab-01b2-4202-8c75-6894e157686e" containerID="b4e305d9cdada99faaafa39a311fbedec0f7aaf54bd6fecddfc69b01446866cb" exitCode=0
Jul 30 04:16:42.690443 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.690169    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"
Jul 30 04:16:42.690443 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.690207    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt" event={"ID":"568901ab-01b2-4202-8c75-6894e157686e","Type":"ContainerDied","Data":"b4e305d9cdada99faaafa39a311fbedec0f7aaf54bd6fecddfc69b01446866cb"}
Jul 30 04:16:42.690443 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.690256    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt" event={"ID":"568901ab-01b2-4202-8c75-6894e157686e","Type":"ContainerDied","Data":"9b58ecc071079bd395b7f131cedea468097378c733d3349585cad394d90d31f6"}
Jul 30 04:16:42.698735 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.698710    2569 scope.go:117] "RemoveContainer" containerID="80d111625636d69df40f06a91f5313a4b80a4b5ae69f92c90cfcecbb6c7808de"
Jul 30 04:16:42.699040 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:16:42.699012    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80d111625636d69df40f06a91f5313a4b80a4b5ae69f92c90cfcecbb6c7808de\": container with ID starting with 80d111625636d69df40f06a91f5313a4b80a4b5ae69f92c90cfcecbb6c7808de not found: ID does not exist" containerID="80d111625636d69df40f06a91f5313a4b80a4b5ae69f92c90cfcecbb6c7808de"
Jul 30 04:16:42.699122 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.699052    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d111625636d69df40f06a91f5313a4b80a4b5ae69f92c90cfcecbb6c7808de"} err="failed to get container status \"80d111625636d69df40f06a91f5313a4b80a4b5ae69f92c90cfcecbb6c7808de\": rpc error: code = NotFound desc = could not find container \"80d111625636d69df40f06a91f5313a4b80a4b5ae69f92c90cfcecbb6c7808de\": container with ID starting with 80d111625636d69df40f06a91f5313a4b80a4b5ae69f92c90cfcecbb6c7808de not found: ID does not exist"
Jul 30 04:16:42.699122 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.699078    2569 scope.go:117] "RemoveContainer" containerID="b4e305d9cdada99faaafa39a311fbedec0f7aaf54bd6fecddfc69b01446866cb"
Jul 30 04:16:42.707252 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.707234    2569 scope.go:117] "RemoveContainer" containerID="b4e305d9cdada99faaafa39a311fbedec0f7aaf54bd6fecddfc69b01446866cb"
Jul 30 04:16:42.707476 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:16:42.707461    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4e305d9cdada99faaafa39a311fbedec0f7aaf54bd6fecddfc69b01446866cb\": container with ID starting with b4e305d9cdada99faaafa39a311fbedec0f7aaf54bd6fecddfc69b01446866cb not found: ID does not exist" containerID="b4e305d9cdada99faaafa39a311fbedec0f7aaf54bd6fecddfc69b01446866cb"
Jul 30 04:16:42.707520 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.707484    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4e305d9cdada99faaafa39a311fbedec0f7aaf54bd6fecddfc69b01446866cb"} err="failed to get container status \"b4e305d9cdada99faaafa39a311fbedec0f7aaf54bd6fecddfc69b01446866cb\": rpc error: code = NotFound desc = could not find container \"b4e305d9cdada99faaafa39a311fbedec0f7aaf54bd6fecddfc69b01446866cb\": container with ID starting with b4e305d9cdada99faaafa39a311fbedec0f7aaf54bd6fecddfc69b01446866cb not found: ID does not exist"
Jul 30 04:16:42.712750 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.712730    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"]
Jul 30 04:16:42.720069 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.720016    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b51d8f16-bccc6949-qzd7m"]
Jul 30 04:16:42.728686 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.728661    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"]
Jul 30 04:16:42.734084 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:42.734066    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b51d8f16-6697f5d674-zvckt"]
Jul 30 04:16:43.379099 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:43.379073    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:43.465959 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:43.465930    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="568901ab-01b2-4202-8c75-6894e157686e" path="/var/lib/kubelet/pods/568901ab-01b2-4202-8c75-6894e157686e/volumes"
Jul 30 04:16:43.466301 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:43.466288    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b767e2cf-6d19-4785-9f0e-a917b06e4c45" path="/var/lib/kubelet/pods/b767e2cf-6d19-4785-9f0e-a917b06e4c45/volumes"
Jul 30 04:16:43.610035 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:43.609999    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:43.856083 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:43.856051    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:44.189492 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:44.189406    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:44.448099 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:44.448011    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:46.569812 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:46.569778    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:46.669682 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:46.669654    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:47.682542 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.682490    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"]
Jul 30 04:16:47.682995 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.682968    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="568901ab-01b2-4202-8c75-6894e157686e" containerName="payload-processing"
Jul 30 04:16:47.682995 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.682986    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="568901ab-01b2-4202-8c75-6894e157686e" containerName="payload-processing"
Jul 30 04:16:47.683108 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.683016    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b767e2cf-6d19-4785-9f0e-a917b06e4c45" containerName="maas-api"
Jul 30 04:16:47.683108 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.683025    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b767e2cf-6d19-4785-9f0e-a917b06e4c45" containerName="maas-api"
Jul 30 04:16:47.683210 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.683117    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="568901ab-01b2-4202-8c75-6894e157686e" containerName="payload-processing"
Jul 30 04:16:47.683210 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.683128    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b767e2cf-6d19-4785-9f0e-a917b06e4c45" containerName="maas-api"
Jul 30 04:16:47.687524 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.687502    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:16:47.689633 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.689609    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lzvw2\""
Jul 30 04:16:47.689732 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.689609    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-484ee705-serving-cert\""
Jul 30 04:16:47.696600 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.696576    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"]
Jul 30 04:16:47.713056 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.713025    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:47.806973 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.806940    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7783a42-8150-4460-9a2e-26c26f20a775-maas-api-tls\") pod \"maas-api-e2e-derive-484ee705-655546c9cc-7bxlq\" (UID: \"a7783a42-8150-4460-9a2e-26c26f20a775\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:16:47.807100 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.806979    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7783a42-8150-4460-9a2e-26c26f20a775-openshift-service-ca\") pod \"maas-api-e2e-derive-484ee705-655546c9cc-7bxlq\" (UID: \"a7783a42-8150-4460-9a2e-26c26f20a775\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:16:47.807100 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.807003    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjvdl\" (UniqueName: \"kubernetes.io/projected/a7783a42-8150-4460-9a2e-26c26f20a775-kube-api-access-hjvdl\") pod \"maas-api-e2e-derive-484ee705-655546c9cc-7bxlq\" (UID: \"a7783a42-8150-4460-9a2e-26c26f20a775\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:16:47.861511 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.861481    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:47.908340 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.908317    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7783a42-8150-4460-9a2e-26c26f20a775-maas-api-tls\") pod \"maas-api-e2e-derive-484ee705-655546c9cc-7bxlq\" (UID: \"a7783a42-8150-4460-9a2e-26c26f20a775\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:16:47.908465 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.908348    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7783a42-8150-4460-9a2e-26c26f20a775-openshift-service-ca\") pod \"maas-api-e2e-derive-484ee705-655546c9cc-7bxlq\" (UID: \"a7783a42-8150-4460-9a2e-26c26f20a775\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:16:47.908465 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.908373    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjvdl\" (UniqueName: \"kubernetes.io/projected/a7783a42-8150-4460-9a2e-26c26f20a775-kube-api-access-hjvdl\") pod \"maas-api-e2e-derive-484ee705-655546c9cc-7bxlq\" (UID: \"a7783a42-8150-4460-9a2e-26c26f20a775\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:16:47.909092 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.909043    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7783a42-8150-4460-9a2e-26c26f20a775-openshift-service-ca\") pod \"maas-api-e2e-derive-484ee705-655546c9cc-7bxlq\" (UID: \"a7783a42-8150-4460-9a2e-26c26f20a775\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:16:47.910988 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.910964    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7783a42-8150-4460-9a2e-26c26f20a775-maas-api-tls\") pod \"maas-api-e2e-derive-484ee705-655546c9cc-7bxlq\" (UID: \"a7783a42-8150-4460-9a2e-26c26f20a775\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:16:47.915567 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.915518    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjvdl\" (UniqueName: \"kubernetes.io/projected/a7783a42-8150-4460-9a2e-26c26f20a775-kube-api-access-hjvdl\") pod \"maas-api-e2e-derive-484ee705-655546c9cc-7bxlq\" (UID: \"a7783a42-8150-4460-9a2e-26c26f20a775\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:16:47.999147 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:47.999090    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:16:48.135359 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.135328    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"]
Jul 30 04:16:48.137329 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:16:48.137302    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7783a42_8150_4460_9a2e_26c26f20a775.slice/crio-d9ba2e42a5b0b6092a3954e0cec2e4f88bf4b21dac1ae4aec2d922cf15a58beb WatchSource:0}: Error finding container d9ba2e42a5b0b6092a3954e0cec2e4f88bf4b21dac1ae4aec2d922cf15a58beb: Status 404 returned error can't find the container with id d9ba2e42a5b0b6092a3954e0cec2e4f88bf4b21dac1ae4aec2d922cf15a58beb
Jul 30 04:16:48.190944 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.190917    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:48.294550 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.294511    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:48.301856 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.301829    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"]
Jul 30 04:16:48.306344 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.306327    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"
Jul 30 04:16:48.309726 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.309705    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-484ee705-dockercfg-zxvwd\""
Jul 30 04:16:48.309829 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.309754    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-484ee705\""
Jul 30 04:16:48.321850 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.321827    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"]
Jul 30 04:16:48.323828 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.323809    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"]
Jul 30 04:16:48.326958 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.326942    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"
Jul 30 04:16:48.337036 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.337015    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"]
Jul 30 04:16:48.411719 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.411687    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76993c7d-e740-490b-bf08-97953fbda59e-plugins-config-volume\") pod \"payload-processing-e2e-derive-484ee705-6764457fbb-x9fks\" (UID: \"76993c7d-e740-490b-bf08-97953fbda59e\") " pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"
Jul 30 04:16:48.411875 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.411740    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c65246f0-4e2e-42a9-84ba-ee5a538bb23c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq\" (UID: \"c65246f0-4e2e-42a9-84ba-ee5a538bb23c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"
Jul 30 04:16:48.411875 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.411812    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5bdp\" (UniqueName: \"kubernetes.io/projected/c65246f0-4e2e-42a9-84ba-ee5a538bb23c-kube-api-access-v5bdp\") pod \"payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq\" (UID: \"c65246f0-4e2e-42a9-84ba-ee5a538bb23c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"
Jul 30 04:16:48.411875 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.411854    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z4gg\" (UniqueName: \"kubernetes.io/projected/76993c7d-e740-490b-bf08-97953fbda59e-kube-api-access-8z4gg\") pod \"payload-processing-e2e-derive-484ee705-6764457fbb-x9fks\" (UID: \"76993c7d-e740-490b-bf08-97953fbda59e\") " pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"
Jul 30 04:16:48.512736 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.512712    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76993c7d-e740-490b-bf08-97953fbda59e-plugins-config-volume\") pod \"payload-processing-e2e-derive-484ee705-6764457fbb-x9fks\" (UID: \"76993c7d-e740-490b-bf08-97953fbda59e\") " pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"
Jul 30 04:16:48.512858 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.512757    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c65246f0-4e2e-42a9-84ba-ee5a538bb23c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq\" (UID: \"c65246f0-4e2e-42a9-84ba-ee5a538bb23c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"
Jul 30 04:16:48.512858 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.512843    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5bdp\" (UniqueName: \"kubernetes.io/projected/c65246f0-4e2e-42a9-84ba-ee5a538bb23c-kube-api-access-v5bdp\") pod \"payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq\" (UID: \"c65246f0-4e2e-42a9-84ba-ee5a538bb23c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"
Jul 30 04:16:48.513097 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.512884    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z4gg\" (UniqueName: \"kubernetes.io/projected/76993c7d-e740-490b-bf08-97953fbda59e-kube-api-access-8z4gg\") pod \"payload-processing-e2e-derive-484ee705-6764457fbb-x9fks\" (UID: \"76993c7d-e740-490b-bf08-97953fbda59e\") " pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"
Jul 30 04:16:48.513651 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.513628    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76993c7d-e740-490b-bf08-97953fbda59e-plugins-config-volume\") pod \"payload-processing-e2e-derive-484ee705-6764457fbb-x9fks\" (UID: \"76993c7d-e740-490b-bf08-97953fbda59e\") " pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"
Jul 30 04:16:48.513972 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.513946    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c65246f0-4e2e-42a9-84ba-ee5a538bb23c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq\" (UID: \"c65246f0-4e2e-42a9-84ba-ee5a538bb23c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"
Jul 30 04:16:48.520597 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.520576    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5bdp\" (UniqueName: \"kubernetes.io/projected/c65246f0-4e2e-42a9-84ba-ee5a538bb23c-kube-api-access-v5bdp\") pod \"payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq\" (UID: \"c65246f0-4e2e-42a9-84ba-ee5a538bb23c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"
Jul 30 04:16:48.521031 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.521007    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z4gg\" (UniqueName: \"kubernetes.io/projected/76993c7d-e740-490b-bf08-97953fbda59e-kube-api-access-8z4gg\") pod \"payload-processing-e2e-derive-484ee705-6764457fbb-x9fks\" (UID: \"76993c7d-e740-490b-bf08-97953fbda59e\") " pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"
Jul 30 04:16:48.616987 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.616917    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"
Jul 30 04:16:48.638197 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.638155    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"
Jul 30 04:16:48.730506 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.730468    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq" event={"ID":"a7783a42-8150-4460-9a2e-26c26f20a775","Type":"ContainerStarted","Data":"eda473dcf4a981356b0170cf77902247d341821d1920460c5596d8c250a10345"}
Jul 30 04:16:48.730965 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.730557    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq" event={"ID":"a7783a42-8150-4460-9a2e-26c26f20a775","Type":"ContainerStarted","Data":"d9ba2e42a5b0b6092a3954e0cec2e4f88bf4b21dac1ae4aec2d922cf15a58beb"}
Jul 30 04:16:48.730965 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.730605    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:16:48.747019 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.746919    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq" podStartSLOduration=1.404750674 podStartE2EDuration="1.746902221s" totalImagesPullingTime="342.151547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:16:48.744756332 +0000 UTC m=+1907.783467368" watchObservedRunningTime="2026-07-30 04:16:48.746902221 +0000 UTC m=+1907.785613242"
Jul 30 04:16:48.766319 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.766272    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"]
Jul 30 04:16:48.770292 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:16:48.770265    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc65246f0_4e2e_42a9_84ba_ee5a538bb23c.slice/crio-2d48d6dcdaf4949cca1e64c3988dc7f1ce19b14e122201510d3f6c0d9f9e72be WatchSource:0}: Error finding container 2d48d6dcdaf4949cca1e64c3988dc7f1ce19b14e122201510d3f6c0d9f9e72be: Status 404 returned error can't find the container with id 2d48d6dcdaf4949cca1e64c3988dc7f1ce19b14e122201510d3f6c0d9f9e72be
Jul 30 04:16:48.998939 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:48.998909    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"]
Jul 30 04:16:49.000664 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:16:49.000639    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76993c7d_e740_490b_bf08_97953fbda59e.slice/crio-d14f27e93d8f249163030cc232d0f5504df985589487b3faeae2c7fca7ac5e92 WatchSource:0}: Error finding container d14f27e93d8f249163030cc232d0f5504df985589487b3faeae2c7fca7ac5e92: Status 404 returned error can't find the container with id d14f27e93d8f249163030cc232d0f5504df985589487b3faeae2c7fca7ac5e92
Jul 30 04:16:49.738708 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:49.738668    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq" event={"ID":"c65246f0-4e2e-42a9-84ba-ee5a538bb23c","Type":"ContainerStarted","Data":"19e9f0b80f4d37dc381c0ea5733b4173f8cc0d6959bceb2a024fe8d99e358952"}
Jul 30 04:16:49.739118 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:49.738731    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq" event={"ID":"c65246f0-4e2e-42a9-84ba-ee5a538bb23c","Type":"ContainerStarted","Data":"2d48d6dcdaf4949cca1e64c3988dc7f1ce19b14e122201510d3f6c0d9f9e72be"}
Jul 30 04:16:49.739118 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:49.738763    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"
Jul 30 04:16:49.740354 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:49.740317    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks" event={"ID":"76993c7d-e740-490b-bf08-97953fbda59e","Type":"ContainerStarted","Data":"916702810aceb6d309365036c88e436c69085752236edca2eeebf0516f3217ff"}
Jul 30 04:16:49.740460 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:49.740376    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks" event={"ID":"76993c7d-e740-490b-bf08-97953fbda59e","Type":"ContainerStarted","Data":"d14f27e93d8f249163030cc232d0f5504df985589487b3faeae2c7fca7ac5e92"}
Jul 30 04:16:49.740460 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:49.740456    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"
Jul 30 04:16:49.754064 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:49.754021    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq" podStartSLOduration=1.754006651 podStartE2EDuration="1.754006651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:16:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:16:49.752446888 +0000 UTC m=+1908.791157917" watchObservedRunningTime="2026-07-30 04:16:49.754006651 +0000 UTC m=+1908.792717669"
Jul 30 04:16:49.768059 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:49.768017    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks" podStartSLOduration=1.768005403 podStartE2EDuration="1.768005403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:16:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:16:49.766228055 +0000 UTC m=+1908.804939075" watchObservedRunningTime="2026-07-30 04:16:49.768005403 +0000 UTC m=+1908.806716473"
Jul 30 04:16:54.745760 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:16:54.745729    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:17:00.746704 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:00.746670    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"
Jul 30 04:17:00.747196 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:00.746751    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"
Jul 30 04:17:16.639304 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.639267    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"]
Jul 30 04:17:16.639764 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.639517    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq" podUID="a7783a42-8150-4460-9a2e-26c26f20a775" containerName="maas-api" containerID="cri-o://eda473dcf4a981356b0170cf77902247d341821d1920460c5596d8c250a10345" gracePeriod=30
Jul 30 04:17:16.667832 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.667799    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:16.696572 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.696525    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"]
Jul 30 04:17:16.696823 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.696795    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks" podUID="76993c7d-e740-490b-bf08-97953fbda59e" containerName="payload-processing" containerID="cri-o://916702810aceb6d309365036c88e436c69085752236edca2eeebf0516f3217ff" gracePeriod=30
Jul 30 04:17:16.704464 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.704431    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"]
Jul 30 04:17:16.704709 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.704682    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq" podUID="c65246f0-4e2e-42a9-84ba-ee5a538bb23c" containerName="payload-pre-processing" containerID="cri-o://19e9f0b80f4d37dc381c0ea5733b4173f8cc0d6959bceb2a024fe8d99e358952" gracePeriod=30
Jul 30 04:17:16.733667 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.733641    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:16.795611 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.795582    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:16.855588 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.855551    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:16.889704 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.889576    2569 generic.go:356] "Generic (PLEG): container finished" podID="a7783a42-8150-4460-9a2e-26c26f20a775" containerID="eda473dcf4a981356b0170cf77902247d341821d1920460c5596d8c250a10345" exitCode=0
Jul 30 04:17:16.889704 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.889652    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq" event={"ID":"a7783a42-8150-4460-9a2e-26c26f20a775","Type":"ContainerDied","Data":"eda473dcf4a981356b0170cf77902247d341821d1920460c5596d8c250a10345"}
Jul 30 04:17:16.892001 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.891974    2569 generic.go:356] "Generic (PLEG): container finished" podID="76993c7d-e740-490b-bf08-97953fbda59e" containerID="916702810aceb6d309365036c88e436c69085752236edca2eeebf0516f3217ff" exitCode=0
Jul 30 04:17:16.892129 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.892093    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks" event={"ID":"76993c7d-e740-490b-bf08-97953fbda59e","Type":"ContainerDied","Data":"916702810aceb6d309365036c88e436c69085752236edca2eeebf0516f3217ff"}
Jul 30 04:17:16.894020 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.893993    2569 generic.go:356] "Generic (PLEG): container finished" podID="c65246f0-4e2e-42a9-84ba-ee5a538bb23c" containerID="19e9f0b80f4d37dc381c0ea5733b4173f8cc0d6959bceb2a024fe8d99e358952" exitCode=0
Jul 30 04:17:16.894116 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.894047    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq" event={"ID":"c65246f0-4e2e-42a9-84ba-ee5a538bb23c","Type":"ContainerDied","Data":"19e9f0b80f4d37dc381c0ea5733b4173f8cc0d6959bceb2a024fe8d99e358952"}
Jul 30 04:17:16.972166 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.971838    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:17:16.977864 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:16.977828    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"
Jul 30 04:17:17.012598 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.012558    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:17.019946 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.019924    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"
Jul 30 04:17:17.048717 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.048690    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjvdl\" (UniqueName: \"kubernetes.io/projected/a7783a42-8150-4460-9a2e-26c26f20a775-kube-api-access-hjvdl\") pod \"a7783a42-8150-4460-9a2e-26c26f20a775\" (UID: \"a7783a42-8150-4460-9a2e-26c26f20a775\") "
Jul 30 04:17:17.048866 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.048759    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7783a42-8150-4460-9a2e-26c26f20a775-maas-api-tls\") pod \"a7783a42-8150-4460-9a2e-26c26f20a775\" (UID: \"a7783a42-8150-4460-9a2e-26c26f20a775\") "
Jul 30 04:17:17.048866 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.048800    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7783a42-8150-4460-9a2e-26c26f20a775-openshift-service-ca\") pod \"a7783a42-8150-4460-9a2e-26c26f20a775\" (UID: \"a7783a42-8150-4460-9a2e-26c26f20a775\") "
Jul 30 04:17:17.048866 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.048839    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76993c7d-e740-490b-bf08-97953fbda59e-plugins-config-volume\") pod \"76993c7d-e740-490b-bf08-97953fbda59e\" (UID: \"76993c7d-e740-490b-bf08-97953fbda59e\") "
Jul 30 04:17:17.049035 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.048903    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z4gg\" (UniqueName: \"kubernetes.io/projected/76993c7d-e740-490b-bf08-97953fbda59e-kube-api-access-8z4gg\") pod \"76993c7d-e740-490b-bf08-97953fbda59e\" (UID: \"76993c7d-e740-490b-bf08-97953fbda59e\") "
Jul 30 04:17:17.049254 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.049223    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76993c7d-e740-490b-bf08-97953fbda59e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "76993c7d-e740-490b-bf08-97953fbda59e" (UID: "76993c7d-e740-490b-bf08-97953fbda59e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:17:17.049254 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.049222    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7783a42-8150-4460-9a2e-26c26f20a775-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a7783a42-8150-4460-9a2e-26c26f20a775" (UID: "a7783a42-8150-4460-9a2e-26c26f20a775"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:17:17.051023 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.050996    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7783a42-8150-4460-9a2e-26c26f20a775-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a7783a42-8150-4460-9a2e-26c26f20a775" (UID: "a7783a42-8150-4460-9a2e-26c26f20a775"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:17:17.051153 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.051132    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76993c7d-e740-490b-bf08-97953fbda59e-kube-api-access-8z4gg" (OuterVolumeSpecName: "kube-api-access-8z4gg") pod "76993c7d-e740-490b-bf08-97953fbda59e" (UID: "76993c7d-e740-490b-bf08-97953fbda59e"). InnerVolumeSpecName "kube-api-access-8z4gg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:17:17.051193 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.051162    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7783a42-8150-4460-9a2e-26c26f20a775-kube-api-access-hjvdl" (OuterVolumeSpecName: "kube-api-access-hjvdl") pod "a7783a42-8150-4460-9a2e-26c26f20a775" (UID: "a7783a42-8150-4460-9a2e-26c26f20a775"). InnerVolumeSpecName "kube-api-access-hjvdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:17:17.150388 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.150314    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c65246f0-4e2e-42a9-84ba-ee5a538bb23c-plugins-config-volume\") pod \"c65246f0-4e2e-42a9-84ba-ee5a538bb23c\" (UID: \"c65246f0-4e2e-42a9-84ba-ee5a538bb23c\") "
Jul 30 04:17:17.150526 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.150393    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5bdp\" (UniqueName: \"kubernetes.io/projected/c65246f0-4e2e-42a9-84ba-ee5a538bb23c-kube-api-access-v5bdp\") pod \"c65246f0-4e2e-42a9-84ba-ee5a538bb23c\" (UID: \"c65246f0-4e2e-42a9-84ba-ee5a538bb23c\") "
Jul 30 04:17:17.150676 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.150659    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z4gg\" (UniqueName: \"kubernetes.io/projected/76993c7d-e740-490b-bf08-97953fbda59e-kube-api-access-8z4gg\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:17:17.150733 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.150681    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjvdl\" (UniqueName: \"kubernetes.io/projected/a7783a42-8150-4460-9a2e-26c26f20a775-kube-api-access-hjvdl\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:17:17.150733 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.150680    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c65246f0-4e2e-42a9-84ba-ee5a538bb23c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c65246f0-4e2e-42a9-84ba-ee5a538bb23c" (UID: "c65246f0-4e2e-42a9-84ba-ee5a538bb23c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:17:17.150733 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.150696    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7783a42-8150-4460-9a2e-26c26f20a775-maas-api-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:17:17.150889 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.150734    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7783a42-8150-4460-9a2e-26c26f20a775-openshift-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:17:17.150889 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.150751    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76993c7d-e740-490b-bf08-97953fbda59e-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:17:17.152686 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.152658    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c65246f0-4e2e-42a9-84ba-ee5a538bb23c-kube-api-access-v5bdp" (OuterVolumeSpecName: "kube-api-access-v5bdp") pod "c65246f0-4e2e-42a9-84ba-ee5a538bb23c" (UID: "c65246f0-4e2e-42a9-84ba-ee5a538bb23c"). InnerVolumeSpecName "kube-api-access-v5bdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:17:17.247820 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.247779    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:17.251850 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.251835    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5bdp\" (UniqueName: \"kubernetes.io/projected/c65246f0-4e2e-42a9-84ba-ee5a538bb23c-kube-api-access-v5bdp\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:17:17.251907 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.251853    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c65246f0-4e2e-42a9-84ba-ee5a538bb23c-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:17:17.740744 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.740627    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:17.843817 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.843788    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:17.900514 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.900479    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq" event={"ID":"c65246f0-4e2e-42a9-84ba-ee5a538bb23c","Type":"ContainerDied","Data":"2d48d6dcdaf4949cca1e64c3988dc7f1ce19b14e122201510d3f6c0d9f9e72be"}
Jul 30 04:17:17.900693 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.900515    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"
Jul 30 04:17:17.900693 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.900573    2569 scope.go:117] "RemoveContainer" containerID="19e9f0b80f4d37dc381c0ea5733b4173f8cc0d6959bceb2a024fe8d99e358952"
Jul 30 04:17:17.901979 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.901954    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq" event={"ID":"a7783a42-8150-4460-9a2e-26c26f20a775","Type":"ContainerDied","Data":"d9ba2e42a5b0b6092a3954e0cec2e4f88bf4b21dac1ae4aec2d922cf15a58beb"}
Jul 30 04:17:17.902046 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.901965    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"
Jul 30 04:17:17.903399 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.903372    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks" event={"ID":"76993c7d-e740-490b-bf08-97953fbda59e","Type":"ContainerDied","Data":"d14f27e93d8f249163030cc232d0f5504df985589487b3faeae2c7fca7ac5e92"}
Jul 30 04:17:17.903563 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.903409    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"
Jul 30 04:17:17.910691 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.910672    2569 scope.go:117] "RemoveContainer" containerID="eda473dcf4a981356b0170cf77902247d341821d1920460c5596d8c250a10345"
Jul 30 04:17:17.918358 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.918334    2569 scope.go:117] "RemoveContainer" containerID="916702810aceb6d309365036c88e436c69085752236edca2eeebf0516f3217ff"
Jul 30 04:17:17.927702 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.927667    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"]
Jul 30 04:17:17.929770 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.929749    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-484ee705-6764457fbb-x9fks"]
Jul 30 04:17:17.931512 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.931488    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:17.941790 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.941766    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"]
Jul 30 04:17:17.945912 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.945892    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-484ee705-659677b8c6-54nvq"]
Jul 30 04:17:17.956272 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.956245    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"]
Jul 30 04:17:17.958161 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:17.958144    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-484ee705-655546c9cc-7bxlq"]
Jul 30 04:17:19.465490 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:19.465458    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76993c7d-e740-490b-bf08-97953fbda59e" path="/var/lib/kubelet/pods/76993c7d-e740-490b-bf08-97953fbda59e/volumes"
Jul 30 04:17:19.465869 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:19.465846    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7783a42-8150-4460-9a2e-26c26f20a775" path="/var/lib/kubelet/pods/a7783a42-8150-4460-9a2e-26c26f20a775/volumes"
Jul 30 04:17:19.466149 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:19.466136    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c65246f0-4e2e-42a9-84ba-ee5a538bb23c" path="/var/lib/kubelet/pods/c65246f0-4e2e-42a9-84ba-ee5a538bb23c/volumes"
Jul 30 04:17:39.928720 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:39.928690    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:46.275404 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:46.275371    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:46.384654 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:46.384625    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:46.476457 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:46.476430    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:46.577208 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:46.577142    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:47.080081 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:17:47.080046    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:13.114615 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:18:13.114579    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:19.927816 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:18:19.927779    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:20.032198 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:18:20.032165    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:20.127793 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:18:20.127759    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:36.064558 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:18:36.064491    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:41.624238 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:18:41.624208    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:41.685992 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:18:41.685957    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:41.767816 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:18:41.767774    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:41.868371 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:18:41.868341    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:52.419320 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:18:52.419284    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:56.498329 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:18:56.498294    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:03.483913 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:03.483885    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:09.775174 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:09.775136    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:09.877015 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:09.876982    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:09.975860 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:09.975829    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:10.075845 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:10.075767    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:10.742646 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:10.742605    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:10.841248 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:10.841214    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:10.941515 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:10.941490    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:12.493853 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:12.493821    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:18.025247 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:18.025211    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:19.135095 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:19.135056    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:24.306156 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:24.306076    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:24.402800 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:24.402768    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:24.602453 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:24.602373    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:24.880082 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:24.879991    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:25.075333 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:25.075304    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:27.920300 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:27.920273    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:28.020204 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:28.020170    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:29.117607 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.117565    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"]
Jul 30 04:19:29.118021 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.117958    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7783a42-8150-4460-9a2e-26c26f20a775" containerName="maas-api"
Jul 30 04:19:29.118021 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.117970    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7783a42-8150-4460-9a2e-26c26f20a775" containerName="maas-api"
Jul 30 04:19:29.118021 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.117977    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c65246f0-4e2e-42a9-84ba-ee5a538bb23c" containerName="payload-pre-processing"
Jul 30 04:19:29.118021 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.117987    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c65246f0-4e2e-42a9-84ba-ee5a538bb23c" containerName="payload-pre-processing"
Jul 30 04:19:29.118021 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.118013    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76993c7d-e740-490b-bf08-97953fbda59e" containerName="payload-processing"
Jul 30 04:19:29.118021 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.118019    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76993c7d-e740-490b-bf08-97953fbda59e" containerName="payload-processing"
Jul 30 04:19:29.118212 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.118096    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c65246f0-4e2e-42a9-84ba-ee5a538bb23c" containerName="payload-pre-processing"
Jul 30 04:19:29.118212 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.118106    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7783a42-8150-4460-9a2e-26c26f20a775" containerName="maas-api"
Jul 30 04:19:29.118212 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.118113    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="76993c7d-e740-490b-bf08-97953fbda59e" containerName="payload-processing"
Jul 30 04:19:29.126324 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.126292    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:19:29.128784 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.128588    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9a3256f1-serving-cert\""
Jul 30 04:19:29.129271 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.129247    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lzvw2\""
Jul 30 04:19:29.131873 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.131850    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"]
Jul 30 04:19:29.151371 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.151347    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:29.261801 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.261765    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ed9c22-6095-422a-a66b-847c0bb315ea-openshift-service-ca\") pod \"maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf\" (UID: \"44ed9c22-6095-422a-a66b-847c0bb315ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:19:29.261979 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.261845    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44ed9c22-6095-422a-a66b-847c0bb315ea-maas-api-tls\") pod \"maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf\" (UID: \"44ed9c22-6095-422a-a66b-847c0bb315ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:19:29.261979 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.261872    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhqx7\" (UniqueName: \"kubernetes.io/projected/44ed9c22-6095-422a-a66b-847c0bb315ea-kube-api-access-dhqx7\") pod \"maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf\" (UID: \"44ed9c22-6095-422a-a66b-847c0bb315ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:19:29.363009 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.362981    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ed9c22-6095-422a-a66b-847c0bb315ea-openshift-service-ca\") pod \"maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf\" (UID: \"44ed9c22-6095-422a-a66b-847c0bb315ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:19:29.363180 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.363022    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44ed9c22-6095-422a-a66b-847c0bb315ea-maas-api-tls\") pod \"maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf\" (UID: \"44ed9c22-6095-422a-a66b-847c0bb315ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:19:29.363180 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.363037    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhqx7\" (UniqueName: \"kubernetes.io/projected/44ed9c22-6095-422a-a66b-847c0bb315ea-kube-api-access-dhqx7\") pod \"maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf\" (UID: \"44ed9c22-6095-422a-a66b-847c0bb315ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:19:29.363180 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:19:29.363149    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-serving-cert: secret "maas-api-e2e-mt-9a3256f1-serving-cert" not found
Jul 30 04:19:29.363297 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:19:29.363221    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44ed9c22-6095-422a-a66b-847c0bb315ea-maas-api-tls podName:44ed9c22-6095-422a-a66b-847c0bb315ea nodeName:}" failed. No retries permitted until 2026-07-30 04:19:29.863203854 +0000 UTC m=+2068.901914851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/44ed9c22-6095-422a-a66b-847c0bb315ea-maas-api-tls") pod "maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf" (UID: "44ed9c22-6095-422a-a66b-847c0bb315ea") : secret "maas-api-e2e-mt-9a3256f1-serving-cert" not found
Jul 30 04:19:29.363618 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.363598    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ed9c22-6095-422a-a66b-847c0bb315ea-openshift-service-ca\") pod \"maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf\" (UID: \"44ed9c22-6095-422a-a66b-847c0bb315ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:19:29.373156 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.373100    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhqx7\" (UniqueName: \"kubernetes.io/projected/44ed9c22-6095-422a-a66b-847c0bb315ea-kube-api-access-dhqx7\") pod \"maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf\" (UID: \"44ed9c22-6095-422a-a66b-847c0bb315ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:19:29.562341 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.562316    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:29.662637 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.662547    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:29.749845 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.749809    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:29.869204 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.869171    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44ed9c22-6095-422a-a66b-847c0bb315ea-maas-api-tls\") pod \"maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf\" (UID: \"44ed9c22-6095-422a-a66b-847c0bb315ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:19:29.871759 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:29.871739    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44ed9c22-6095-422a-a66b-847c0bb315ea-maas-api-tls\") pod \"maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf\" (UID: \"44ed9c22-6095-422a-a66b-847c0bb315ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:19:30.038311 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:30.038281    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:19:30.169283 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:30.169259    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"]
Jul 30 04:19:30.171745 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:19:30.171712    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44ed9c22_6095_422a_a66b_847c0bb315ea.slice/crio-6425bfeb500f6edceabd8ed67fc1a5a04d12a144bbff63b7ee3ea1198a4c16de WatchSource:0}: Error finding container 6425bfeb500f6edceabd8ed67fc1a5a04d12a144bbff63b7ee3ea1198a4c16de: Status 404 returned error can't find the container with id 6425bfeb500f6edceabd8ed67fc1a5a04d12a144bbff63b7ee3ea1198a4c16de
Jul 30 04:19:30.549416 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:30.549380    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf" event={"ID":"44ed9c22-6095-422a-a66b-847c0bb315ea","Type":"ContainerStarted","Data":"6425bfeb500f6edceabd8ed67fc1a5a04d12a144bbff63b7ee3ea1198a4c16de"}
Jul 30 04:19:31.556454 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:31.556408    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf" event={"ID":"44ed9c22-6095-422a-a66b-847c0bb315ea","Type":"ContainerStarted","Data":"21f3d2ee9c2e7eba981abe113a340367251385c31add25afec55106c9203f093"}
Jul 30 04:19:31.556896 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:31.556558    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:19:31.571431 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:31.571370    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf" podStartSLOduration=2.016505573 podStartE2EDuration="2.571353712s" totalImagesPullingTime="554.848139ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:19:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:19:31.570769696 +0000 UTC m=+2070.609480717" watchObservedRunningTime="2026-07-30 04:19:31.571353712 +0000 UTC m=+2070.610064732"
Jul 30 04:19:37.567049 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:37.567020    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:19:52.902407 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:52.902373    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:53.587062 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:53.587029    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:53.685017 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:53.684987    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:54.020342 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:54.020315    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:59.640707 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:59.640668    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:59.702969 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:19:59.702941    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:01.579627 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:01.579600    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:20:01.587162 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:01.587141    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:20:10.506092 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.506060    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"]
Jul 30 04:20:10.506498 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.506246    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf" podUID="44ed9c22-6095-422a-a66b-847c0bb315ea" containerName="maas-api" containerID="cri-o://21f3d2ee9c2e7eba981abe113a340367251385c31add25afec55106c9203f093" gracePeriod=30
Jul 30 04:20:10.521909 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.521884    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:10.599255 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.599227    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:10.657321 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.657287    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:10.713395 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.713365    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:10.750117 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.750090    2569 generic.go:356] "Generic (PLEG): container finished" podID="44ed9c22-6095-422a-a66b-847c0bb315ea" containerID="21f3d2ee9c2e7eba981abe113a340367251385c31add25afec55106c9203f093" exitCode=0
Jul 30 04:20:10.750240 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.750162    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf" event={"ID":"44ed9c22-6095-422a-a66b-847c0bb315ea","Type":"ContainerDied","Data":"21f3d2ee9c2e7eba981abe113a340367251385c31add25afec55106c9203f093"}
Jul 30 04:20:10.750240 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.750213    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf" event={"ID":"44ed9c22-6095-422a-a66b-847c0bb315ea","Type":"ContainerDied","Data":"6425bfeb500f6edceabd8ed67fc1a5a04d12a144bbff63b7ee3ea1198a4c16de"}
Jul 30 04:20:10.750240 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.750231    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6425bfeb500f6edceabd8ed67fc1a5a04d12a144bbff63b7ee3ea1198a4c16de"
Jul 30 04:20:10.751016 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.751002    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:20:10.787097 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.787072    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ed9c22-6095-422a-a66b-847c0bb315ea-openshift-service-ca\") pod \"44ed9c22-6095-422a-a66b-847c0bb315ea\" (UID: \"44ed9c22-6095-422a-a66b-847c0bb315ea\") "
Jul 30 04:20:10.787097 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.787099    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhqx7\" (UniqueName: \"kubernetes.io/projected/44ed9c22-6095-422a-a66b-847c0bb315ea-kube-api-access-dhqx7\") pod \"44ed9c22-6095-422a-a66b-847c0bb315ea\" (UID: \"44ed9c22-6095-422a-a66b-847c0bb315ea\") "
Jul 30 04:20:10.787292 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.787129    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44ed9c22-6095-422a-a66b-847c0bb315ea-maas-api-tls\") pod \"44ed9c22-6095-422a-a66b-847c0bb315ea\" (UID: \"44ed9c22-6095-422a-a66b-847c0bb315ea\") "
Jul 30 04:20:10.787399 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.787371    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44ed9c22-6095-422a-a66b-847c0bb315ea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "44ed9c22-6095-422a-a66b-847c0bb315ea" (UID: "44ed9c22-6095-422a-a66b-847c0bb315ea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:20:10.789314 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.789288    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44ed9c22-6095-422a-a66b-847c0bb315ea-kube-api-access-dhqx7" (OuterVolumeSpecName: "kube-api-access-dhqx7") pod "44ed9c22-6095-422a-a66b-847c0bb315ea" (UID: "44ed9c22-6095-422a-a66b-847c0bb315ea"). InnerVolumeSpecName "kube-api-access-dhqx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:20:10.789314 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.789294    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44ed9c22-6095-422a-a66b-847c0bb315ea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "44ed9c22-6095-422a-a66b-847c0bb315ea" (UID: "44ed9c22-6095-422a-a66b-847c0bb315ea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:20:10.887833 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.887802    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ed9c22-6095-422a-a66b-847c0bb315ea-openshift-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:20:10.887833 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.887834    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhqx7\" (UniqueName: \"kubernetes.io/projected/44ed9c22-6095-422a-a66b-847c0bb315ea-kube-api-access-dhqx7\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:20:10.888033 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:10.887848    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44ed9c22-6095-422a-a66b-847c0bb315ea-maas-api-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:20:11.069956 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:11.069877    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:11.754971 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:11.754888    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"
Jul 30 04:20:11.771736 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:11.771706    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"]
Jul 30 04:20:11.778836 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:11.778807    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9a3256f1-d4b489c7d-hbtzf"]
Jul 30 04:20:13.464735 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:13.464701    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44ed9c22-6095-422a-a66b-847c0bb315ea" path="/var/lib/kubelet/pods/44ed9c22-6095-422a-a66b-847c0bb315ea/volumes"
Jul 30 04:20:16.693254 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:16.693226    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:16.794790 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:16.794758    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:16.960518 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:16.960443    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:17.195290 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:17.195263    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:17.329202 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:17.329170    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:17.391223 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:17.391194    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:17.583191 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:17.583096    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:23.154162 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:23.154118    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:23.260269 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:23.260240    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:23.353210 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:23.353178    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:23.508149 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:23.508067    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:29.102605 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:29.102567    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:29.206246 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:20:29.206219    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:21:37.748904 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:21:37.748858    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:21:51.461114 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:21:51.461074    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:21:51.514937 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:21:51.514905    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:42.843514 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:42.843429    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:43.157830 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:43.157759    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:43.313760 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:43.313732    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:45.986731 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:45.986700    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:46.085750 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:46.085720    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:47.077892 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.077860    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"]
Jul 30 04:22:47.078352 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.078333    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44ed9c22-6095-422a-a66b-847c0bb315ea" containerName="maas-api"
Jul 30 04:22:47.078352 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.078350    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="44ed9c22-6095-422a-a66b-847c0bb315ea" containerName="maas-api"
Jul 30 04:22:47.078465 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.078447    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="44ed9c22-6095-422a-a66b-847c0bb315ea" containerName="maas-api"
Jul 30 04:22:47.081559 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.081518    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:22:47.084319 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.084295    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lzvw2\""
Jul 30 04:22:47.084486 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.084471    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-bd1da342-serving-cert\""
Jul 30 04:22:47.092407 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.092382    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"]
Jul 30 04:22:47.122189 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.122159    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:47.215384 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.215355    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f709c0c-f24b-419c-9d42-44a526f991cd-maas-api-tls\") pod \"maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456\" (UID: \"1f709c0c-f24b-419c-9d42-44a526f991cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:22:47.215384 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.215387    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f709c0c-f24b-419c-9d42-44a526f991cd-openshift-service-ca\") pod \"maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456\" (UID: \"1f709c0c-f24b-419c-9d42-44a526f991cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:22:47.215582 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.215456    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmn88\" (UniqueName: \"kubernetes.io/projected/1f709c0c-f24b-419c-9d42-44a526f991cd-kube-api-access-bmn88\") pod \"maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456\" (UID: \"1f709c0c-f24b-419c-9d42-44a526f991cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:22:47.316800 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.316770    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f709c0c-f24b-419c-9d42-44a526f991cd-maas-api-tls\") pod \"maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456\" (UID: \"1f709c0c-f24b-419c-9d42-44a526f991cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:22:47.316800 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.316802    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f709c0c-f24b-419c-9d42-44a526f991cd-openshift-service-ca\") pod \"maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456\" (UID: \"1f709c0c-f24b-419c-9d42-44a526f991cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:22:47.316993 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.316848    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmn88\" (UniqueName: \"kubernetes.io/projected/1f709c0c-f24b-419c-9d42-44a526f991cd-kube-api-access-bmn88\") pod \"maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456\" (UID: \"1f709c0c-f24b-419c-9d42-44a526f991cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:22:47.316993 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:22:47.316925    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-serving-cert: secret "maas-api-e2e-mt-bd1da342-serving-cert" not found
Jul 30 04:22:47.317079 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:22:47.317002    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f709c0c-f24b-419c-9d42-44a526f991cd-maas-api-tls podName:1f709c0c-f24b-419c-9d42-44a526f991cd nodeName:}" failed. No retries permitted until 2026-07-30 04:22:47.816985346 +0000 UTC m=+2266.855696343 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1f709c0c-f24b-419c-9d42-44a526f991cd-maas-api-tls") pod "maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456" (UID: "1f709c0c-f24b-419c-9d42-44a526f991cd") : secret "maas-api-e2e-mt-bd1da342-serving-cert" not found
Jul 30 04:22:47.317419 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.317402    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f709c0c-f24b-419c-9d42-44a526f991cd-openshift-service-ca\") pod \"maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456\" (UID: \"1f709c0c-f24b-419c-9d42-44a526f991cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:22:47.324697 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.324674    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmn88\" (UniqueName: \"kubernetes.io/projected/1f709c0c-f24b-419c-9d42-44a526f991cd-kube-api-access-bmn88\") pod \"maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456\" (UID: \"1f709c0c-f24b-419c-9d42-44a526f991cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:22:47.633072 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.633040    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:47.736820 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.736783    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:47.751671 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.751639    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"]
Jul 30 04:22:47.755092 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.755073    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"
Jul 30 04:22:47.761344 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.761318    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-bd1da342-dockercfg-zx6rm\""
Jul 30 04:22:47.761456 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.761356    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-bd1da342\""
Jul 30 04:22:47.768505 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.768482    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"]
Jul 30 04:22:47.820988 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.820962    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6f0ce90-09e9-4f17-be2e-d1598b1870c3-plugins-config-volume\") pod \"payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5\" (UID: \"e6f0ce90-09e9-4f17-be2e-d1598b1870c3\") " pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"
Jul 30 04:22:47.821141 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.821044    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8btnj\" (UniqueName: \"kubernetes.io/projected/e6f0ce90-09e9-4f17-be2e-d1598b1870c3-kube-api-access-8btnj\") pod \"payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5\" (UID: \"e6f0ce90-09e9-4f17-be2e-d1598b1870c3\") " pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"
Jul 30 04:22:47.821141 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.821076    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f709c0c-f24b-419c-9d42-44a526f991cd-maas-api-tls\") pod \"maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456\" (UID: \"1f709c0c-f24b-419c-9d42-44a526f991cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:22:47.823498 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.823477    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f709c0c-f24b-419c-9d42-44a526f991cd-maas-api-tls\") pod \"maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456\" (UID: \"1f709c0c-f24b-419c-9d42-44a526f991cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:22:47.921494 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.921432    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8btnj\" (UniqueName: \"kubernetes.io/projected/e6f0ce90-09e9-4f17-be2e-d1598b1870c3-kube-api-access-8btnj\") pod \"payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5\" (UID: \"e6f0ce90-09e9-4f17-be2e-d1598b1870c3\") " pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"
Jul 30 04:22:47.921494 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.921485    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6f0ce90-09e9-4f17-be2e-d1598b1870c3-plugins-config-volume\") pod \"payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5\" (UID: \"e6f0ce90-09e9-4f17-be2e-d1598b1870c3\") " pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"
Jul 30 04:22:47.922201 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.922183    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6f0ce90-09e9-4f17-be2e-d1598b1870c3-plugins-config-volume\") pod \"payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5\" (UID: \"e6f0ce90-09e9-4f17-be2e-d1598b1870c3\") " pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"
Jul 30 04:22:47.929753 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.929736    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8btnj\" (UniqueName: \"kubernetes.io/projected/e6f0ce90-09e9-4f17-be2e-d1598b1870c3-kube-api-access-8btnj\") pod \"payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5\" (UID: \"e6f0ce90-09e9-4f17-be2e-d1598b1870c3\") " pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"
Jul 30 04:22:47.993582 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:47.993561    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:22:48.065722 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:48.065694    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"
Jul 30 04:22:48.132220 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:48.131943    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"]
Jul 30 04:22:48.135129 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:48.135112    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 04:22:48.209977 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:48.209953    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"]
Jul 30 04:22:48.211711 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:22:48.211684    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6f0ce90_09e9_4f17_be2e_d1598b1870c3.slice/crio-ed4d57a4543e0d81630b7fd123574111da43429ea246950fc3854abdf9b77a5b WatchSource:0}: Error finding container ed4d57a4543e0d81630b7fd123574111da43429ea246950fc3854abdf9b77a5b: Status 404 returned error can't find the container with id ed4d57a4543e0d81630b7fd123574111da43429ea246950fc3854abdf9b77a5b
Jul 30 04:22:48.510502 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:48.510423    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5" event={"ID":"e6f0ce90-09e9-4f17-be2e-d1598b1870c3","Type":"ContainerStarted","Data":"6667ebd03cb4d794d530bf6f51449c08d7faa872741450d0ed8b80a543b5f3cf"}
Jul 30 04:22:48.510502 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:48.510472    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5" event={"ID":"e6f0ce90-09e9-4f17-be2e-d1598b1870c3","Type":"ContainerStarted","Data":"ed4d57a4543e0d81630b7fd123574111da43429ea246950fc3854abdf9b77a5b"}
Jul 30 04:22:48.510746 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:48.510512    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"
Jul 30 04:22:48.511722 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:48.511696    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456" event={"ID":"1f709c0c-f24b-419c-9d42-44a526f991cd","Type":"ContainerStarted","Data":"92d35415361513dd8fbfc86542932a104d5a80e161fa51ce90ff1111bd24b310"}
Jul 30 04:22:48.527497 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:48.527445    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5" podStartSLOduration=1.527429139 podStartE2EDuration="1.527429139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:22:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:22:48.526606105 +0000 UTC m=+2267.565317124" watchObservedRunningTime="2026-07-30 04:22:48.527429139 +0000 UTC m=+2267.566140160"
Jul 30 04:22:49.518949 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:49.518916    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456" event={"ID":"1f709c0c-f24b-419c-9d42-44a526f991cd","Type":"ContainerStarted","Data":"c7b11941e6c8190a0daffd788ae1b29c27947bba2137abbe53cdaa208b2186d6"}
Jul 30 04:22:49.519320 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:49.518989    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:22:49.537311 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:49.537258    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456" podStartSLOduration=2.067168805 podStartE2EDuration="2.537240237s" totalImagesPullingTime="470.071432ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:22:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:22:49.536314975 +0000 UTC m=+2268.575026005" watchObservedRunningTime="2026-07-30 04:22:49.537240237 +0000 UTC m=+2268.575951256"
Jul 30 04:22:55.530112 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:55.530084    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:22:56.911891 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:56.911857    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:59.521084 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:22:59.521052    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"
Jul 30 04:23:11.010197 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:23:11.010162    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:23:20.369139 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:23:20.369104    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:10.511599 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:10.511565    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:10.612599 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:10.612572    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:11.254061 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.254021    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"]
Jul 30 04:24:11.254245 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.254207    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456" podUID="1f709c0c-f24b-419c-9d42-44a526f991cd" containerName="maas-api" containerID="cri-o://c7b11941e6c8190a0daffd788ae1b29c27947bba2137abbe53cdaa208b2186d6" gracePeriod=30
Jul 30 04:24:11.273666 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.273642    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:11.301491 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.301459    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"]
Jul 30 04:24:11.301752 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.301725    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5" podUID="e6f0ce90-09e9-4f17-be2e-d1598b1870c3" containerName="payload-processing" containerID="cri-o://6667ebd03cb4d794d530bf6f51449c08d7faa872741450d0ed8b80a543b5f3cf" gracePeriod=30
Jul 30 04:24:11.371234 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.371206    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:11.439521 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.439492    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:11.503483 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.503452    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:11.545765 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.545743    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:24:11.575254 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.575231    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"
Jul 30 04:24:11.716062 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.716018    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmn88\" (UniqueName: \"kubernetes.io/projected/1f709c0c-f24b-419c-9d42-44a526f991cd-kube-api-access-bmn88\") pod \"1f709c0c-f24b-419c-9d42-44a526f991cd\" (UID: \"1f709c0c-f24b-419c-9d42-44a526f991cd\") "
Jul 30 04:24:11.716253 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.716114    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f709c0c-f24b-419c-9d42-44a526f991cd-maas-api-tls\") pod \"1f709c0c-f24b-419c-9d42-44a526f991cd\" (UID: \"1f709c0c-f24b-419c-9d42-44a526f991cd\") "
Jul 30 04:24:11.716253 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.716144    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f709c0c-f24b-419c-9d42-44a526f991cd-openshift-service-ca\") pod \"1f709c0c-f24b-419c-9d42-44a526f991cd\" (UID: \"1f709c0c-f24b-419c-9d42-44a526f991cd\") "
Jul 30 04:24:11.716253 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.716177    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8btnj\" (UniqueName: \"kubernetes.io/projected/e6f0ce90-09e9-4f17-be2e-d1598b1870c3-kube-api-access-8btnj\") pod \"e6f0ce90-09e9-4f17-be2e-d1598b1870c3\" (UID: \"e6f0ce90-09e9-4f17-be2e-d1598b1870c3\") "
Jul 30 04:24:11.716253 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.716243    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6f0ce90-09e9-4f17-be2e-d1598b1870c3-plugins-config-volume\") pod \"e6f0ce90-09e9-4f17-be2e-d1598b1870c3\" (UID: \"e6f0ce90-09e9-4f17-be2e-d1598b1870c3\") "
Jul 30 04:24:11.716520 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.716493    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f709c0c-f24b-419c-9d42-44a526f991cd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1f709c0c-f24b-419c-9d42-44a526f991cd" (UID: "1f709c0c-f24b-419c-9d42-44a526f991cd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:24:11.716763 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.716737    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6f0ce90-09e9-4f17-be2e-d1598b1870c3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6f0ce90-09e9-4f17-be2e-d1598b1870c3" (UID: "e6f0ce90-09e9-4f17-be2e-d1598b1870c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:24:11.718506 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.718485    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f709c0c-f24b-419c-9d42-44a526f991cd-kube-api-access-bmn88" (OuterVolumeSpecName: "kube-api-access-bmn88") pod "1f709c0c-f24b-419c-9d42-44a526f991cd" (UID: "1f709c0c-f24b-419c-9d42-44a526f991cd"). InnerVolumeSpecName "kube-api-access-bmn88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:24:11.718699 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.718675    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f709c0c-f24b-419c-9d42-44a526f991cd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1f709c0c-f24b-419c-9d42-44a526f991cd" (UID: "1f709c0c-f24b-419c-9d42-44a526f991cd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:24:11.718699 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.718683    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6f0ce90-09e9-4f17-be2e-d1598b1870c3-kube-api-access-8btnj" (OuterVolumeSpecName: "kube-api-access-8btnj") pod "e6f0ce90-09e9-4f17-be2e-d1598b1870c3" (UID: "e6f0ce90-09e9-4f17-be2e-d1598b1870c3"). InnerVolumeSpecName "kube-api-access-8btnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:24:11.817278 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.817208    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f709c0c-f24b-419c-9d42-44a526f991cd-maas-api-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:24:11.817278 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.817232    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f709c0c-f24b-419c-9d42-44a526f991cd-openshift-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:24:11.817278 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.817242    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8btnj\" (UniqueName: \"kubernetes.io/projected/e6f0ce90-09e9-4f17-be2e-d1598b1870c3-kube-api-access-8btnj\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:24:11.817278 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.817252    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6f0ce90-09e9-4f17-be2e-d1598b1870c3-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:24:11.817278 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.817262    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmn88\" (UniqueName: \"kubernetes.io/projected/1f709c0c-f24b-419c-9d42-44a526f991cd-kube-api-access-bmn88\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:24:11.937242 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.937209    2569 generic.go:356] "Generic (PLEG): container finished" podID="e6f0ce90-09e9-4f17-be2e-d1598b1870c3" containerID="6667ebd03cb4d794d530bf6f51449c08d7faa872741450d0ed8b80a543b5f3cf" exitCode=0
Jul 30 04:24:11.937414 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.937254    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"
Jul 30 04:24:11.937414 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.937283    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5" event={"ID":"e6f0ce90-09e9-4f17-be2e-d1598b1870c3","Type":"ContainerDied","Data":"6667ebd03cb4d794d530bf6f51449c08d7faa872741450d0ed8b80a543b5f3cf"}
Jul 30 04:24:11.937414 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.937328    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5" event={"ID":"e6f0ce90-09e9-4f17-be2e-d1598b1870c3","Type":"ContainerDied","Data":"ed4d57a4543e0d81630b7fd123574111da43429ea246950fc3854abdf9b77a5b"}
Jul 30 04:24:11.937414 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.937357    2569 scope.go:117] "RemoveContainer" containerID="6667ebd03cb4d794d530bf6f51449c08d7faa872741450d0ed8b80a543b5f3cf"
Jul 30 04:24:11.938629 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.938609    2569 generic.go:356] "Generic (PLEG): container finished" podID="1f709c0c-f24b-419c-9d42-44a526f991cd" containerID="c7b11941e6c8190a0daffd788ae1b29c27947bba2137abbe53cdaa208b2186d6" exitCode=0
Jul 30 04:24:11.938719 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.938665    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"
Jul 30 04:24:11.938719 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.938697    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456" event={"ID":"1f709c0c-f24b-419c-9d42-44a526f991cd","Type":"ContainerDied","Data":"c7b11941e6c8190a0daffd788ae1b29c27947bba2137abbe53cdaa208b2186d6"}
Jul 30 04:24:11.938833 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.938728    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456" event={"ID":"1f709c0c-f24b-419c-9d42-44a526f991cd","Type":"ContainerDied","Data":"92d35415361513dd8fbfc86542932a104d5a80e161fa51ce90ff1111bd24b310"}
Jul 30 04:24:11.947809 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.947794    2569 scope.go:117] "RemoveContainer" containerID="6667ebd03cb4d794d530bf6f51449c08d7faa872741450d0ed8b80a543b5f3cf"
Jul 30 04:24:11.948078 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:24:11.948061    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6667ebd03cb4d794d530bf6f51449c08d7faa872741450d0ed8b80a543b5f3cf\": container with ID starting with 6667ebd03cb4d794d530bf6f51449c08d7faa872741450d0ed8b80a543b5f3cf not found: ID does not exist" containerID="6667ebd03cb4d794d530bf6f51449c08d7faa872741450d0ed8b80a543b5f3cf"
Jul 30 04:24:11.948121 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.948087    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6667ebd03cb4d794d530bf6f51449c08d7faa872741450d0ed8b80a543b5f3cf"} err="failed to get container status \"6667ebd03cb4d794d530bf6f51449c08d7faa872741450d0ed8b80a543b5f3cf\": rpc error: code = NotFound desc = could not find container \"6667ebd03cb4d794d530bf6f51449c08d7faa872741450d0ed8b80a543b5f3cf\": container with ID starting with 6667ebd03cb4d794d530bf6f51449c08d7faa872741450d0ed8b80a543b5f3cf not found: ID does not exist"
Jul 30 04:24:11.948121 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.948103    2569 scope.go:117] "RemoveContainer" containerID="c7b11941e6c8190a0daffd788ae1b29c27947bba2137abbe53cdaa208b2186d6"
Jul 30 04:24:11.955971 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.955943    2569 scope.go:117] "RemoveContainer" containerID="c7b11941e6c8190a0daffd788ae1b29c27947bba2137abbe53cdaa208b2186d6"
Jul 30 04:24:11.956204 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:24:11.956187    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7b11941e6c8190a0daffd788ae1b29c27947bba2137abbe53cdaa208b2186d6\": container with ID starting with c7b11941e6c8190a0daffd788ae1b29c27947bba2137abbe53cdaa208b2186d6 not found: ID does not exist" containerID="c7b11941e6c8190a0daffd788ae1b29c27947bba2137abbe53cdaa208b2186d6"
Jul 30 04:24:11.956262 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.956210    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b11941e6c8190a0daffd788ae1b29c27947bba2137abbe53cdaa208b2186d6"} err="failed to get container status \"c7b11941e6c8190a0daffd788ae1b29c27947bba2137abbe53cdaa208b2186d6\": rpc error: code = NotFound desc = could not find container \"c7b11941e6c8190a0daffd788ae1b29c27947bba2137abbe53cdaa208b2186d6\": container with ID starting with c7b11941e6c8190a0daffd788ae1b29c27947bba2137abbe53cdaa208b2186d6 not found: ID does not exist"
Jul 30 04:24:11.959105 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.959086    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"]
Jul 30 04:24:11.962182 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.962157    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bd1da342-79dc8dc8bb-ptdq5"]
Jul 30 04:24:11.971153 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.971134    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"]
Jul 30 04:24:11.974616 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:11.974598    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd1da342-c5d6b4cc8-jq456"]
Jul 30 04:24:13.465601 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:13.465568    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f709c0c-f24b-419c-9d42-44a526f991cd" path="/var/lib/kubelet/pods/1f709c0c-f24b-419c-9d42-44a526f991cd/volumes"
Jul 30 04:24:13.465983 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:13.465914    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6f0ce90-09e9-4f17-be2e-d1598b1870c3" path="/var/lib/kubelet/pods/e6f0ce90-09e9-4f17-be2e-d1598b1870c3/volumes"
Jul 30 04:24:14.701500 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:14.701468    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:14.835324 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:14.835298    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:19.810485 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:19.810451    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:56.449213 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:56.449181    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:56.697578 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:24:56.697543    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:01.611469 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:01.611443    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:25:01.618841 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:01.618822    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:25:02.107267 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:02.107235    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:02.163725 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:02.163693    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:02.376180 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:02.376078    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:02.614619 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:02.614589    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:24.166259 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:24.166178    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:24.266273 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:24.266239    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:24.366596 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:24.366571    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:24.980729 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:24.980699    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:25.093303 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:25.093243    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:25.181279 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:25.181236    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:26.449580 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:26.449526    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:32.063625 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:32.063583    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:32.165187 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:32.165153    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:32.725667 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:32.725634    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:38.543389 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:38.543354    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:38.662614 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:38.662580    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:38.783578 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:38.783548    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:38.876082 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:38.875968    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:49.296612 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:49.296564    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:49.537353 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:25:49.537318    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:01.721521 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:01.721491    2569 scope.go:117] "RemoveContainer" containerID="21f3d2ee9c2e7eba981abe113a340367251385c31add25afec55106c9203f093"
Jul 30 04:26:10.527790 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:10.527746    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:16.971910 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:16.971870    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:17.070673 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:17.070643    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:17.320997 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:17.320969    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:17.651624 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:17.651517    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:17.804925 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:17.804875    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:20.118811 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:20.118779    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:20.223391 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:20.223363    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:21.217824 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.217785    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"]
Jul 30 04:26:21.218437 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.218416    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f709c0c-f24b-419c-9d42-44a526f991cd" containerName="maas-api"
Jul 30 04:26:21.218557 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.218440    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f709c0c-f24b-419c-9d42-44a526f991cd" containerName="maas-api"
Jul 30 04:26:21.218557 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.218467    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f0ce90-09e9-4f17-be2e-d1598b1870c3" containerName="payload-processing"
Jul 30 04:26:21.218557 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.218477    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f0ce90-09e9-4f17-be2e-d1598b1870c3" containerName="payload-processing"
Jul 30 04:26:21.218723 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.218593    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6f0ce90-09e9-4f17-be2e-d1598b1870c3" containerName="payload-processing"
Jul 30 04:26:21.218723 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.218614    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f709c0c-f24b-419c-9d42-44a526f991cd" containerName="maas-api"
Jul 30 04:26:21.221443 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.221423    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:26:21.223451 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.223422    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-9acc4f-serving-cert\""
Jul 30 04:26:21.223620 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.223603    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lzvw2\""
Jul 30 04:26:21.232360 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.232335    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"]
Jul 30 04:26:21.256411 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.256384    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:21.325593 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.325566    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad9fe0f-cb0f-407d-b61d-894195e32f86-maas-api-tls\") pod \"maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm\" (UID: \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:26:21.325711 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.325607    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv57v\" (UniqueName: \"kubernetes.io/projected/5ad9fe0f-cb0f-407d-b61d-894195e32f86-kube-api-access-nv57v\") pod \"maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm\" (UID: \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:26:21.325711 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.325633    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad9fe0f-cb0f-407d-b61d-894195e32f86-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm\" (UID: \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:26:21.426019 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.425984    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad9fe0f-cb0f-407d-b61d-894195e32f86-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm\" (UID: \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:26:21.426201 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.426071    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad9fe0f-cb0f-407d-b61d-894195e32f86-maas-api-tls\") pod \"maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm\" (UID: \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:26:21.426201 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.426094    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv57v\" (UniqueName: \"kubernetes.io/projected/5ad9fe0f-cb0f-407d-b61d-894195e32f86-kube-api-access-nv57v\") pod \"maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm\" (UID: \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:26:21.426324 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:26:21.426226    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-serving-cert: secret "maas-api-e2e-inf-a-9acc4f-serving-cert" not found
Jul 30 04:26:21.426324 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:26:21.426298    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad9fe0f-cb0f-407d-b61d-894195e32f86-maas-api-tls podName:5ad9fe0f-cb0f-407d-b61d-894195e32f86 nodeName:}" failed. No retries permitted until 2026-07-30 04:26:21.926278831 +0000 UTC m=+2480.964989849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5ad9fe0f-cb0f-407d-b61d-894195e32f86-maas-api-tls") pod "maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm" (UID: "5ad9fe0f-cb0f-407d-b61d-894195e32f86") : secret "maas-api-e2e-inf-a-9acc4f-serving-cert" not found
Jul 30 04:26:21.426644 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.426626    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad9fe0f-cb0f-407d-b61d-894195e32f86-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm\" (UID: \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:26:21.444222 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.444200    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv57v\" (UniqueName: \"kubernetes.io/projected/5ad9fe0f-cb0f-407d-b61d-894195e32f86-kube-api-access-nv57v\") pod \"maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm\" (UID: \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:26:21.487033 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.486973    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:21.744552 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.744458    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:21.847907 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.847874    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:21.930731 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.930692    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad9fe0f-cb0f-407d-b61d-894195e32f86-maas-api-tls\") pod \"maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm\" (UID: \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:26:21.933178 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:21.933146    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad9fe0f-cb0f-407d-b61d-894195e32f86-maas-api-tls\") pod \"maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm\" (UID: \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:26:22.132943 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:22.132915    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:26:22.262818 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:22.262784    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"]
Jul 30 04:26:22.587759 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:22.587727    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm" event={"ID":"5ad9fe0f-cb0f-407d-b61d-894195e32f86","Type":"ContainerStarted","Data":"2aeb8674c92849e767dc981c443d71a23b7fbfc995f1cef8815341e43851cee8"}
Jul 30 04:26:23.594014 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:23.593970    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm" event={"ID":"5ad9fe0f-cb0f-407d-b61d-894195e32f86","Type":"ContainerStarted","Data":"17b19d5133570e6f084a6194b958d239886aba59d75ff086b1dc9aabc5519e96"}
Jul 30 04:26:23.594452 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:23.594037    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:26:23.608942 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:23.608882    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm" podStartSLOduration=2.156492294 podStartE2EDuration="2.608862894s" totalImagesPullingTime="452.3706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:26:23.608286805 +0000 UTC m=+2482.646997825" watchObservedRunningTime="2026-07-30 04:26:23.608862894 +0000 UTC m=+2482.647573916"
Jul 30 04:26:29.605282 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:29.605253    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:26:45.144263 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:45.144230    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:45.495228 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:45.495137    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:45.853704 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:45.853648    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:46.138787 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:46.138699    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:48.279364 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:48.279332    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:48.372039 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:48.372011    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:49.376173 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.376136    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"]
Jul 30 04:26:49.380113 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.380093    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:26:49.382128 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.382106    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c24378-serving-cert\""
Jul 30 04:26:49.389412 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.389391    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"]
Jul 30 04:26:49.410849 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.410824    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:49.456124 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.456096    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgvtq\" (UniqueName: \"kubernetes.io/projected/0c520787-e949-4dc5-adb5-1b943e41df32-kube-api-access-cgvtq\") pod \"maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k\" (UID: \"0c520787-e949-4dc5-adb5-1b943e41df32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:26:49.456208 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.456139    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c520787-e949-4dc5-adb5-1b943e41df32-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k\" (UID: \"0c520787-e949-4dc5-adb5-1b943e41df32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:26:49.456256 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.456203    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c520787-e949-4dc5-adb5-1b943e41df32-maas-api-tls\") pod \"maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k\" (UID: \"0c520787-e949-4dc5-adb5-1b943e41df32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:26:49.556487 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.556456    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c520787-e949-4dc5-adb5-1b943e41df32-maas-api-tls\") pod \"maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k\" (UID: \"0c520787-e949-4dc5-adb5-1b943e41df32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:26:49.556689 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.556527    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgvtq\" (UniqueName: \"kubernetes.io/projected/0c520787-e949-4dc5-adb5-1b943e41df32-kube-api-access-cgvtq\") pod \"maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k\" (UID: \"0c520787-e949-4dc5-adb5-1b943e41df32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:26:49.556689 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.556602    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c520787-e949-4dc5-adb5-1b943e41df32-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k\" (UID: \"0c520787-e949-4dc5-adb5-1b943e41df32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:26:49.556689 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:26:49.556622    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-serving-cert: secret "maas-api-e2e-inf-b-c24378-serving-cert" not found
Jul 30 04:26:49.556689 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:26:49.556674    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c520787-e949-4dc5-adb5-1b943e41df32-maas-api-tls podName:0c520787-e949-4dc5-adb5-1b943e41df32 nodeName:}" failed. No retries permitted until 2026-07-30 04:26:50.056658871 +0000 UTC m=+2509.095369868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0c520787-e949-4dc5-adb5-1b943e41df32-maas-api-tls") pod "maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k" (UID: "0c520787-e949-4dc5-adb5-1b943e41df32") : secret "maas-api-e2e-inf-b-c24378-serving-cert" not found
Jul 30 04:26:49.557244 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.557223    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c520787-e949-4dc5-adb5-1b943e41df32-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k\" (UID: \"0c520787-e949-4dc5-adb5-1b943e41df32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:26:49.566916 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.566897    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgvtq\" (UniqueName: \"kubernetes.io/projected/0c520787-e949-4dc5-adb5-1b943e41df32-kube-api-access-cgvtq\") pod \"maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k\" (UID: \"0c520787-e949-4dc5-adb5-1b943e41df32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:26:49.849855 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.849822    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:49.946818 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:49.946785    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:50.017280 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.017248    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:50.018566 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.018528    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"]
Jul 30 04:26:50.022068 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.022048    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"
Jul 30 04:26:50.024423 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.024402    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-c24378\""
Jul 30 04:26:50.024672 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.024658    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-c24378-dockercfg-pd759\""
Jul 30 04:26:50.033861 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.033838    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"]
Jul 30 04:26:50.038804 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.038784    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"]
Jul 30 04:26:50.041980 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.041962    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"
Jul 30 04:26:50.050491 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.050468    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"]
Jul 30 04:26:50.060613 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.060501    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28p2z\" (UniqueName: \"kubernetes.io/projected/696f4cb7-505f-4d59-8937-74b4a19c0482-kube-api-access-28p2z\") pod \"payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd\" (UID: \"696f4cb7-505f-4d59-8937-74b4a19c0482\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"
Jul 30 04:26:50.060613 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.060568    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c520787-e949-4dc5-adb5-1b943e41df32-maas-api-tls\") pod \"maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k\" (UID: \"0c520787-e949-4dc5-adb5-1b943e41df32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:26:50.060613 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.060618    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/696f4cb7-505f-4d59-8937-74b4a19c0482-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd\" (UID: \"696f4cb7-505f-4d59-8937-74b4a19c0482\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"
Jul 30 04:26:50.060613 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.060653    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d33b4aee-b850-41ea-9588-88cc4b246da1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9\" (UID: \"d33b4aee-b850-41ea-9588-88cc4b246da1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"
Jul 30 04:26:50.061132 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.060683    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjfns\" (UniqueName: \"kubernetes.io/projected/d33b4aee-b850-41ea-9588-88cc4b246da1-kube-api-access-vjfns\") pod \"payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9\" (UID: \"d33b4aee-b850-41ea-9588-88cc4b246da1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"
Jul 30 04:26:50.063652 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.063630    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c520787-e949-4dc5-adb5-1b943e41df32-maas-api-tls\") pod \"maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k\" (UID: \"0c520787-e949-4dc5-adb5-1b943e41df32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:26:50.161592 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.161500    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/696f4cb7-505f-4d59-8937-74b4a19c0482-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd\" (UID: \"696f4cb7-505f-4d59-8937-74b4a19c0482\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"
Jul 30 04:26:50.161592 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.161558    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d33b4aee-b850-41ea-9588-88cc4b246da1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9\" (UID: \"d33b4aee-b850-41ea-9588-88cc4b246da1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"
Jul 30 04:26:50.161592 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.161578    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjfns\" (UniqueName: \"kubernetes.io/projected/d33b4aee-b850-41ea-9588-88cc4b246da1-kube-api-access-vjfns\") pod \"payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9\" (UID: \"d33b4aee-b850-41ea-9588-88cc4b246da1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"
Jul 30 04:26:50.161785 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.161664    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28p2z\" (UniqueName: \"kubernetes.io/projected/696f4cb7-505f-4d59-8937-74b4a19c0482-kube-api-access-28p2z\") pod \"payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd\" (UID: \"696f4cb7-505f-4d59-8937-74b4a19c0482\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"
Jul 30 04:26:50.162207 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.162186    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/696f4cb7-505f-4d59-8937-74b4a19c0482-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd\" (UID: \"696f4cb7-505f-4d59-8937-74b4a19c0482\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"
Jul 30 04:26:50.162381 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.162362    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d33b4aee-b850-41ea-9588-88cc4b246da1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9\" (UID: \"d33b4aee-b850-41ea-9588-88cc4b246da1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"
Jul 30 04:26:50.169865 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.169834    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28p2z\" (UniqueName: \"kubernetes.io/projected/696f4cb7-505f-4d59-8937-74b4a19c0482-kube-api-access-28p2z\") pod \"payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd\" (UID: \"696f4cb7-505f-4d59-8937-74b4a19c0482\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"
Jul 30 04:26:50.169951 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.169867    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjfns\" (UniqueName: \"kubernetes.io/projected/d33b4aee-b850-41ea-9588-88cc4b246da1-kube-api-access-vjfns\") pod \"payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9\" (UID: \"d33b4aee-b850-41ea-9588-88cc4b246da1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"
Jul 30 04:26:50.291392 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.291365    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:26:50.334204 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.334170    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"
Jul 30 04:26:50.353508 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.353473    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"
Jul 30 04:26:50.446665 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.446574    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"]
Jul 30 04:26:50.450056 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:26:50.450029    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c520787_e949_4dc5_adb5_1b943e41df32.slice/crio-d2ee3df2f8716931181f5c17ac46a7cda2fd38e09e127ca57ad89ca0446c67f9 WatchSource:0}: Error finding container d2ee3df2f8716931181f5c17ac46a7cda2fd38e09e127ca57ad89ca0446c67f9: Status 404 returned error can't find the container with id d2ee3df2f8716931181f5c17ac46a7cda2fd38e09e127ca57ad89ca0446c67f9
Jul 30 04:26:50.493212 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.493070    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"]
Jul 30 04:26:50.495490 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:26:50.495459    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod696f4cb7_505f_4d59_8937_74b4a19c0482.slice/crio-5b64e2b93963c8ff0b4d582c4d1c9bb81ba96109df3b4ccfd61adb675ffba820 WatchSource:0}: Error finding container 5b64e2b93963c8ff0b4d582c4d1c9bb81ba96109df3b4ccfd61adb675ffba820: Status 404 returned error can't find the container with id 5b64e2b93963c8ff0b4d582c4d1c9bb81ba96109df3b4ccfd61adb675ffba820
Jul 30 04:26:50.517126 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.517035    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"]
Jul 30 04:26:50.524578 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:26:50.524552    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd33b4aee_b850_41ea_9588_88cc4b246da1.slice/crio-355e89407bbf56d51e7ea175b1c9ef50f4859ba00f1df310f57283055c964cb4 WatchSource:0}: Error finding container 355e89407bbf56d51e7ea175b1c9ef50f4859ba00f1df310f57283055c964cb4: Status 404 returned error can't find the container with id 355e89407bbf56d51e7ea175b1c9ef50f4859ba00f1df310f57283055c964cb4
Jul 30 04:26:50.727303 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.727217    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd" event={"ID":"696f4cb7-505f-4d59-8937-74b4a19c0482","Type":"ContainerStarted","Data":"073c87316dce3e2104c23c3fffa726bb90655cbcf927375c2bad56218f25331b"}
Jul 30 04:26:50.727303 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.727270    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd" event={"ID":"696f4cb7-505f-4d59-8937-74b4a19c0482","Type":"ContainerStarted","Data":"5b64e2b93963c8ff0b4d582c4d1c9bb81ba96109df3b4ccfd61adb675ffba820"}
Jul 30 04:26:50.727507 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.727320    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"
Jul 30 04:26:50.728595 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.728568    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k" event={"ID":"0c520787-e949-4dc5-adb5-1b943e41df32","Type":"ContainerStarted","Data":"d2ee3df2f8716931181f5c17ac46a7cda2fd38e09e127ca57ad89ca0446c67f9"}
Jul 30 04:26:50.729936 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.729912    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9" event={"ID":"d33b4aee-b850-41ea-9588-88cc4b246da1","Type":"ContainerStarted","Data":"129bf451b40fd4e8734b5765f40d3662daecfe781ff04aaf8f6eeedaddedc739"}
Jul 30 04:26:50.730038 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.729950    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9" event={"ID":"d33b4aee-b850-41ea-9588-88cc4b246da1","Type":"ContainerStarted","Data":"355e89407bbf56d51e7ea175b1c9ef50f4859ba00f1df310f57283055c964cb4"}
Jul 30 04:26:50.730082 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.730037    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"
Jul 30 04:26:50.742418 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.742377    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd" podStartSLOduration=0.742363034 podStartE2EDuration="742.363034ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:26:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:26:50.740487936 +0000 UTC m=+2509.779198955" watchObservedRunningTime="2026-07-30 04:26:50.742363034 +0000 UTC m=+2509.781074053"
Jul 30 04:26:50.754905 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:50.754775    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9" podStartSLOduration=0.754761407 podStartE2EDuration="754.761407ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:26:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:26:50.754546748 +0000 UTC m=+2509.793257761" watchObservedRunningTime="2026-07-30 04:26:50.754761407 +0000 UTC m=+2509.793472426"
Jul 30 04:26:51.736946 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:51.736857    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k" event={"ID":"0c520787-e949-4dc5-adb5-1b943e41df32","Type":"ContainerStarted","Data":"82744edcdab3380ce2b7325e13ff2f63f4ee9029f582a4b99c449e1c4a49ed69"}
Jul 30 04:26:51.737368 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:51.737038    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:26:51.753393 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:51.753345    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k" podStartSLOduration=2.244129228 podStartE2EDuration="2.753331093s" totalImagesPullingTime="509.201865ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:26:51.752236453 +0000 UTC m=+2510.790947484" watchObservedRunningTime="2026-07-30 04:26:51.753331093 +0000 UTC m=+2510.792042111"
Jul 30 04:26:57.747549 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:26:57.747507    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:27:01.738875 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:27:01.738836    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"
Jul 30 04:27:01.739239 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:27:01.738997    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"
Jul 30 04:27:13.306249 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:27:13.306211    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:27:28.183000 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:27:28.182969    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:27:30.128030 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:27:30.127998    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:27:39.313498 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:27:39.313465    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:27:39.364455 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:27:39.364425    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:27:48.366571 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:27:48.366527    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:01.913323 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:01.913290    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:01.965925 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:01.965879    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:27.811061 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:27.810955    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:28.701725 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:28.701693    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:34.859185 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:34.859152    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"]
Jul 30 04:28:34.859818 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:34.859357    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm" podUID="5ad9fe0f-cb0f-407d-b61d-894195e32f86" containerName="maas-api" containerID="cri-o://17b19d5133570e6f084a6194b958d239886aba59d75ff086b1dc9aabc5519e96" gracePeriod=30
Jul 30 04:28:34.889906 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:34.889857    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:34.958048 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:34.958014    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:35.016124 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.016090    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:35.117154 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.117058    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:35.270208 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.270177    2569 generic.go:356] "Generic (PLEG): container finished" podID="5ad9fe0f-cb0f-407d-b61d-894195e32f86" containerID="17b19d5133570e6f084a6194b958d239886aba59d75ff086b1dc9aabc5519e96" exitCode=0
Jul 30 04:28:35.270362 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.270240    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm" event={"ID":"5ad9fe0f-cb0f-407d-b61d-894195e32f86","Type":"ContainerDied","Data":"17b19d5133570e6f084a6194b958d239886aba59d75ff086b1dc9aabc5519e96"}
Jul 30 04:28:35.314736 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.314711    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:28:35.409097 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.409020    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad9fe0f-cb0f-407d-b61d-894195e32f86-openshift-service-ca\") pod \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\" (UID: \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\") "
Jul 30 04:28:35.409259 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.409100    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv57v\" (UniqueName: \"kubernetes.io/projected/5ad9fe0f-cb0f-407d-b61d-894195e32f86-kube-api-access-nv57v\") pod \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\" (UID: \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\") "
Jul 30 04:28:35.409335 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.409250    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad9fe0f-cb0f-407d-b61d-894195e32f86-maas-api-tls\") pod \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\" (UID: \"5ad9fe0f-cb0f-407d-b61d-894195e32f86\") "
Jul 30 04:28:35.409436 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.409414    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ad9fe0f-cb0f-407d-b61d-894195e32f86-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5ad9fe0f-cb0f-407d-b61d-894195e32f86" (UID: "5ad9fe0f-cb0f-407d-b61d-894195e32f86"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:28:35.409585 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.409570    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad9fe0f-cb0f-407d-b61d-894195e32f86-openshift-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:28:35.411196 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.411174    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ad9fe0f-cb0f-407d-b61d-894195e32f86-kube-api-access-nv57v" (OuterVolumeSpecName: "kube-api-access-nv57v") pod "5ad9fe0f-cb0f-407d-b61d-894195e32f86" (UID: "5ad9fe0f-cb0f-407d-b61d-894195e32f86"). InnerVolumeSpecName "kube-api-access-nv57v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:28:35.411250 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.411229    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ad9fe0f-cb0f-407d-b61d-894195e32f86-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5ad9fe0f-cb0f-407d-b61d-894195e32f86" (UID: "5ad9fe0f-cb0f-407d-b61d-894195e32f86"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:28:35.510638 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.510612    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv57v\" (UniqueName: \"kubernetes.io/projected/5ad9fe0f-cb0f-407d-b61d-894195e32f86-kube-api-access-nv57v\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:28:35.510638 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.510634    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad9fe0f-cb0f-407d-b61d-894195e32f86-maas-api-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:28:35.583702 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:35.583677    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:36.279209 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:36.279179    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"
Jul 30 04:28:36.279209 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:36.279200    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm" event={"ID":"5ad9fe0f-cb0f-407d-b61d-894195e32f86","Type":"ContainerDied","Data":"2aeb8674c92849e767dc981c443d71a23b7fbfc995f1cef8815341e43851cee8"}
Jul 30 04:28:36.279733 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:36.279271    2569 scope.go:117] "RemoveContainer" containerID="17b19d5133570e6f084a6194b958d239886aba59d75ff086b1dc9aabc5519e96"
Jul 30 04:28:36.294770 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:36.294748    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"]
Jul 30 04:28:36.298778 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:36.298757    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9acc4f-66d7966767-tjpdm"]
Jul 30 04:28:37.465665 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:37.465629    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ad9fe0f-cb0f-407d-b61d-894195e32f86" path="/var/lib/kubelet/pods/5ad9fe0f-cb0f-407d-b61d-894195e32f86/volumes"
Jul 30 04:28:40.636583 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:40.636550    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:46.444725 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:46.444694    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:46.696952 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:46.696857    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:46.930552 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:46.930492    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:57.476246 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.476199    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"]
Jul 30 04:28:57.476719 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.476463    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k" podUID="0c520787-e949-4dc5-adb5-1b943e41df32" containerName="maas-api" containerID="cri-o://82744edcdab3380ce2b7325e13ff2f63f4ee9029f582a4b99c449e1c4a49ed69" gracePeriod=30
Jul 30 04:28:57.503959 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.503936    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:57.534418 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.534392    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"]
Jul 30 04:28:57.534669 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.534646    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9" podUID="d33b4aee-b850-41ea-9588-88cc4b246da1" containerName="payload-processing" containerID="cri-o://129bf451b40fd4e8734b5765f40d3662daecfe781ff04aaf8f6eeedaddedc739" gracePeriod=30
Jul 30 04:28:57.541859 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.541834    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"]
Jul 30 04:28:57.542071 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.542050    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd" podUID="696f4cb7-505f-4d59-8937-74b4a19c0482" containerName="payload-pre-processing" containerID="cri-o://073c87316dce3e2104c23c3fffa726bb90655cbcf927375c2bad56218f25331b" gracePeriod=30
Jul 30 04:28:57.571769 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.571739    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:57.632563 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.632520    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:57.692614 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.692579    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:57.743245 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.743197    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k" podUID="0c520787-e949-4dc5-adb5-1b943e41df32" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.60:8443/health\": dial tcp 10.132.0.60:8443: connect: connection refused"
Jul 30 04:28:57.816744 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.816722    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"
Jul 30 04:28:57.819651 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.819633    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"
Jul 30 04:28:57.839517 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.839496    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:28:57.895887 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.895859    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d33b4aee-b850-41ea-9588-88cc4b246da1-plugins-config-volume\") pod \"d33b4aee-b850-41ea-9588-88cc4b246da1\" (UID: \"d33b4aee-b850-41ea-9588-88cc4b246da1\") "
Jul 30 04:28:57.896025 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.895894    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c520787-e949-4dc5-adb5-1b943e41df32-maas-api-tls\") pod \"0c520787-e949-4dc5-adb5-1b943e41df32\" (UID: \"0c520787-e949-4dc5-adb5-1b943e41df32\") "
Jul 30 04:28:57.896025 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.895918    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28p2z\" (UniqueName: \"kubernetes.io/projected/696f4cb7-505f-4d59-8937-74b4a19c0482-kube-api-access-28p2z\") pod \"696f4cb7-505f-4d59-8937-74b4a19c0482\" (UID: \"696f4cb7-505f-4d59-8937-74b4a19c0482\") "
Jul 30 04:28:57.896025 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.896016    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/696f4cb7-505f-4d59-8937-74b4a19c0482-plugins-config-volume\") pod \"696f4cb7-505f-4d59-8937-74b4a19c0482\" (UID: \"696f4cb7-505f-4d59-8937-74b4a19c0482\") "
Jul 30 04:28:57.896195 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.896049    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c520787-e949-4dc5-adb5-1b943e41df32-openshift-service-ca\") pod \"0c520787-e949-4dc5-adb5-1b943e41df32\" (UID: \"0c520787-e949-4dc5-adb5-1b943e41df32\") "
Jul 30 04:28:57.896195 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.896097    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjfns\" (UniqueName: \"kubernetes.io/projected/d33b4aee-b850-41ea-9588-88cc4b246da1-kube-api-access-vjfns\") pod \"d33b4aee-b850-41ea-9588-88cc4b246da1\" (UID: \"d33b4aee-b850-41ea-9588-88cc4b246da1\") "
Jul 30 04:28:57.896195 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.896128    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgvtq\" (UniqueName: \"kubernetes.io/projected/0c520787-e949-4dc5-adb5-1b943e41df32-kube-api-access-cgvtq\") pod \"0c520787-e949-4dc5-adb5-1b943e41df32\" (UID: \"0c520787-e949-4dc5-adb5-1b943e41df32\") "
Jul 30 04:28:57.896397 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.896373    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/696f4cb7-505f-4d59-8937-74b4a19c0482-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "696f4cb7-505f-4d59-8937-74b4a19c0482" (UID: "696f4cb7-505f-4d59-8937-74b4a19c0482"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:28:57.896456 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.896285    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d33b4aee-b850-41ea-9588-88cc4b246da1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d33b4aee-b850-41ea-9588-88cc4b246da1" (UID: "d33b4aee-b850-41ea-9588-88cc4b246da1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:28:57.896524 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.896456    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c520787-e949-4dc5-adb5-1b943e41df32-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0c520787-e949-4dc5-adb5-1b943e41df32" (UID: "0c520787-e949-4dc5-adb5-1b943e41df32"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:28:57.896688 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.896666    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d33b4aee-b850-41ea-9588-88cc4b246da1-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:28:57.896896 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.896693    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/696f4cb7-505f-4d59-8937-74b4a19c0482-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:28:57.896896 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.896711    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c520787-e949-4dc5-adb5-1b943e41df32-openshift-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:28:57.898376 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.898351    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d33b4aee-b850-41ea-9588-88cc4b246da1-kube-api-access-vjfns" (OuterVolumeSpecName: "kube-api-access-vjfns") pod "d33b4aee-b850-41ea-9588-88cc4b246da1" (UID: "d33b4aee-b850-41ea-9588-88cc4b246da1"). InnerVolumeSpecName "kube-api-access-vjfns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:28:57.898453 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.898380    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c520787-e949-4dc5-adb5-1b943e41df32-kube-api-access-cgvtq" (OuterVolumeSpecName: "kube-api-access-cgvtq") pod "0c520787-e949-4dc5-adb5-1b943e41df32" (UID: "0c520787-e949-4dc5-adb5-1b943e41df32"). InnerVolumeSpecName "kube-api-access-cgvtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:28:57.898652 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.898637    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c520787-e949-4dc5-adb5-1b943e41df32-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0c520787-e949-4dc5-adb5-1b943e41df32" (UID: "0c520787-e949-4dc5-adb5-1b943e41df32"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:28:57.898703 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.898688    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/696f4cb7-505f-4d59-8937-74b4a19c0482-kube-api-access-28p2z" (OuterVolumeSpecName: "kube-api-access-28p2z") pod "696f4cb7-505f-4d59-8937-74b4a19c0482" (UID: "696f4cb7-505f-4d59-8937-74b4a19c0482"). InnerVolumeSpecName "kube-api-access-28p2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:28:57.997717 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.997660    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjfns\" (UniqueName: \"kubernetes.io/projected/d33b4aee-b850-41ea-9588-88cc4b246da1-kube-api-access-vjfns\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:28:57.997717 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.997682    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgvtq\" (UniqueName: \"kubernetes.io/projected/0c520787-e949-4dc5-adb5-1b943e41df32-kube-api-access-cgvtq\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:28:57.997717 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.997691    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c520787-e949-4dc5-adb5-1b943e41df32-maas-api-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:28:57.997717 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:57.997700    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28p2z\" (UniqueName: \"kubernetes.io/projected/696f4cb7-505f-4d59-8937-74b4a19c0482-kube-api-access-28p2z\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:28:58.056805 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.056781    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:28:58.399215 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.399185    2569 generic.go:356] "Generic (PLEG): container finished" podID="696f4cb7-505f-4d59-8937-74b4a19c0482" containerID="073c87316dce3e2104c23c3fffa726bb90655cbcf927375c2bad56218f25331b" exitCode=0
Jul 30 04:28:58.399394 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.399229    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"
Jul 30 04:28:58.399394 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.399263    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd" event={"ID":"696f4cb7-505f-4d59-8937-74b4a19c0482","Type":"ContainerDied","Data":"073c87316dce3e2104c23c3fffa726bb90655cbcf927375c2bad56218f25331b"}
Jul 30 04:28:58.399394 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.399315    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd" event={"ID":"696f4cb7-505f-4d59-8937-74b4a19c0482","Type":"ContainerDied","Data":"5b64e2b93963c8ff0b4d582c4d1c9bb81ba96109df3b4ccfd61adb675ffba820"}
Jul 30 04:28:58.399394 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.399344    2569 scope.go:117] "RemoveContainer" containerID="073c87316dce3e2104c23c3fffa726bb90655cbcf927375c2bad56218f25331b"
Jul 30 04:28:58.400709 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.400686    2569 generic.go:356] "Generic (PLEG): container finished" podID="0c520787-e949-4dc5-adb5-1b943e41df32" containerID="82744edcdab3380ce2b7325e13ff2f63f4ee9029f582a4b99c449e1c4a49ed69" exitCode=0
Jul 30 04:28:58.400803 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.400742    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"
Jul 30 04:28:58.400857 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.400739    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k" event={"ID":"0c520787-e949-4dc5-adb5-1b943e41df32","Type":"ContainerDied","Data":"82744edcdab3380ce2b7325e13ff2f63f4ee9029f582a4b99c449e1c4a49ed69"}
Jul 30 04:28:58.400857 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.400845    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k" event={"ID":"0c520787-e949-4dc5-adb5-1b943e41df32","Type":"ContainerDied","Data":"d2ee3df2f8716931181f5c17ac46a7cda2fd38e09e127ca57ad89ca0446c67f9"}
Jul 30 04:28:58.402146 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.402123    2569 generic.go:356] "Generic (PLEG): container finished" podID="d33b4aee-b850-41ea-9588-88cc4b246da1" containerID="129bf451b40fd4e8734b5765f40d3662daecfe781ff04aaf8f6eeedaddedc739" exitCode=0
Jul 30 04:28:58.402268 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.402165    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"
Jul 30 04:28:58.402268 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.402164    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9" event={"ID":"d33b4aee-b850-41ea-9588-88cc4b246da1","Type":"ContainerDied","Data":"129bf451b40fd4e8734b5765f40d3662daecfe781ff04aaf8f6eeedaddedc739"}
Jul 30 04:28:58.402388 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.402308    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9" event={"ID":"d33b4aee-b850-41ea-9588-88cc4b246da1","Type":"ContainerDied","Data":"355e89407bbf56d51e7ea175b1c9ef50f4859ba00f1df310f57283055c964cb4"}
Jul 30 04:28:58.409676 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.409659    2569 scope.go:117] "RemoveContainer" containerID="073c87316dce3e2104c23c3fffa726bb90655cbcf927375c2bad56218f25331b"
Jul 30 04:28:58.409948 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:28:58.409918    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"073c87316dce3e2104c23c3fffa726bb90655cbcf927375c2bad56218f25331b\": container with ID starting with 073c87316dce3e2104c23c3fffa726bb90655cbcf927375c2bad56218f25331b not found: ID does not exist" containerID="073c87316dce3e2104c23c3fffa726bb90655cbcf927375c2bad56218f25331b"
Jul 30 04:28:58.410056 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.409956    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"073c87316dce3e2104c23c3fffa726bb90655cbcf927375c2bad56218f25331b"} err="failed to get container status \"073c87316dce3e2104c23c3fffa726bb90655cbcf927375c2bad56218f25331b\": rpc error: code = NotFound desc = could not find container \"073c87316dce3e2104c23c3fffa726bb90655cbcf927375c2bad56218f25331b\": container with ID starting with 073c87316dce3e2104c23c3fffa726bb90655cbcf927375c2bad56218f25331b not found: ID does not exist"
Jul 30 04:28:58.410056 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.409980    2569 scope.go:117] "RemoveContainer" containerID="82744edcdab3380ce2b7325e13ff2f63f4ee9029f582a4b99c449e1c4a49ed69"
Jul 30 04:28:58.418003 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.417987    2569 scope.go:117] "RemoveContainer" containerID="82744edcdab3380ce2b7325e13ff2f63f4ee9029f582a4b99c449e1c4a49ed69"
Jul 30 04:28:58.418243 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:28:58.418228    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82744edcdab3380ce2b7325e13ff2f63f4ee9029f582a4b99c449e1c4a49ed69\": container with ID starting with 82744edcdab3380ce2b7325e13ff2f63f4ee9029f582a4b99c449e1c4a49ed69 not found: ID does not exist" containerID="82744edcdab3380ce2b7325e13ff2f63f4ee9029f582a4b99c449e1c4a49ed69"
Jul 30 04:28:58.418289 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.418249    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82744edcdab3380ce2b7325e13ff2f63f4ee9029f582a4b99c449e1c4a49ed69"} err="failed to get container status \"82744edcdab3380ce2b7325e13ff2f63f4ee9029f582a4b99c449e1c4a49ed69\": rpc error: code = NotFound desc = could not find container \"82744edcdab3380ce2b7325e13ff2f63f4ee9029f582a4b99c449e1c4a49ed69\": container with ID starting with 82744edcdab3380ce2b7325e13ff2f63f4ee9029f582a4b99c449e1c4a49ed69 not found: ID does not exist"
Jul 30 04:28:58.418289 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.418262    2569 scope.go:117] "RemoveContainer" containerID="129bf451b40fd4e8734b5765f40d3662daecfe781ff04aaf8f6eeedaddedc739"
Jul 30 04:28:58.421826 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.421807    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"]
Jul 30 04:28:58.426113 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.426086    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c24378-dc85d4648-wq4pd"]
Jul 30 04:28:58.427083 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.427068    2569 scope.go:117] "RemoveContainer" containerID="129bf451b40fd4e8734b5765f40d3662daecfe781ff04aaf8f6eeedaddedc739"
Jul 30 04:28:58.427318 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:28:58.427292    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"129bf451b40fd4e8734b5765f40d3662daecfe781ff04aaf8f6eeedaddedc739\": container with ID starting with 129bf451b40fd4e8734b5765f40d3662daecfe781ff04aaf8f6eeedaddedc739 not found: ID does not exist" containerID="129bf451b40fd4e8734b5765f40d3662daecfe781ff04aaf8f6eeedaddedc739"
Jul 30 04:28:58.427379 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.427327    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"129bf451b40fd4e8734b5765f40d3662daecfe781ff04aaf8f6eeedaddedc739"} err="failed to get container status \"129bf451b40fd4e8734b5765f40d3662daecfe781ff04aaf8f6eeedaddedc739\": rpc error: code = NotFound desc = could not find container \"129bf451b40fd4e8734b5765f40d3662daecfe781ff04aaf8f6eeedaddedc739\": container with ID starting with 129bf451b40fd4e8734b5765f40d3662daecfe781ff04aaf8f6eeedaddedc739 not found: ID does not exist"
Jul 30 04:28:58.435363 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.435344    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"]
Jul 30 04:28:58.437759 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.437741    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c24378-74469dd95d-h5hf9"]
Jul 30 04:28:58.446085 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.446064    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"]
Jul 30 04:28:58.451728 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:58.451708    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c24378-5b7dcb4d49-gtj4k"]
Jul 30 04:28:59.465350 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:59.465319    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c520787-e949-4dc5-adb5-1b943e41df32" path="/var/lib/kubelet/pods/0c520787-e949-4dc5-adb5-1b943e41df32/volumes"
Jul 30 04:28:59.465716 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:59.465665    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="696f4cb7-505f-4d59-8937-74b4a19c0482" path="/var/lib/kubelet/pods/696f4cb7-505f-4d59-8937-74b4a19c0482/volumes"
Jul 30 04:28:59.465963 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:28:59.465950    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d33b4aee-b850-41ea-9588-88cc4b246da1" path="/var/lib/kubelet/pods/d33b4aee-b850-41ea-9588-88cc4b246da1/volumes"
Jul 30 04:29:03.957418 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:03.957380    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:04.058078 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:04.058048    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:04.225344 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:04.225266    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:04.453976 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:04.453949    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:04.793553 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:04.793512    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:05.108498 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:05.108425    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:05.564376 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:05.564343    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:08.051889 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:08.051849    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:08.147877 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:08.147845    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:09.166293 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.166260    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"]
Jul 30 04:29:09.166697 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.166669    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ad9fe0f-cb0f-407d-b61d-894195e32f86" containerName="maas-api"
Jul 30 04:29:09.166697 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.166682    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ad9fe0f-cb0f-407d-b61d-894195e32f86" containerName="maas-api"
Jul 30 04:29:09.166697 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.166690    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d33b4aee-b850-41ea-9588-88cc4b246da1" containerName="payload-processing"
Jul 30 04:29:09.166697 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.166695    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d33b4aee-b850-41ea-9588-88cc4b246da1" containerName="payload-processing"
Jul 30 04:29:09.166826 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.166704    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="696f4cb7-505f-4d59-8937-74b4a19c0482" containerName="payload-pre-processing"
Jul 30 04:29:09.166826 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.166709    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="696f4cb7-505f-4d59-8937-74b4a19c0482" containerName="payload-pre-processing"
Jul 30 04:29:09.166826 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.166715    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c520787-e949-4dc5-adb5-1b943e41df32" containerName="maas-api"
Jul 30 04:29:09.166826 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.166721    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c520787-e949-4dc5-adb5-1b943e41df32" containerName="maas-api"
Jul 30 04:29:09.166826 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.166775    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ad9fe0f-cb0f-407d-b61d-894195e32f86" containerName="maas-api"
Jul 30 04:29:09.166826 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.166785    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="696f4cb7-505f-4d59-8937-74b4a19c0482" containerName="payload-pre-processing"
Jul 30 04:29:09.166826 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.166791    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d33b4aee-b850-41ea-9588-88cc4b246da1" containerName="payload-processing"
Jul 30 04:29:09.166826 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.166797    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c520787-e949-4dc5-adb5-1b943e41df32" containerName="maas-api"
Jul 30 04:29:09.169608 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.169592    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:29:09.171657 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.171635    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lzvw2\""
Jul 30 04:29:09.171790 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.171630    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-eb6c99-serving-cert\""
Jul 30 04:29:09.179459 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.179436    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"]
Jul 30 04:29:09.190149 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.190121    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:09.288224 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.288189    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3478799b-da94-4bdd-b2be-9eaab784591f-openshift-service-ca\") pod \"maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs\" (UID: \"3478799b-da94-4bdd-b2be-9eaab784591f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:29:09.288390 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.288241    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3478799b-da94-4bdd-b2be-9eaab784591f-maas-api-tls\") pod \"maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs\" (UID: \"3478799b-da94-4bdd-b2be-9eaab784591f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:29:09.288390 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.288259    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjc6b\" (UniqueName: \"kubernetes.io/projected/3478799b-da94-4bdd-b2be-9eaab784591f-kube-api-access-mjc6b\") pod \"maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs\" (UID: \"3478799b-da94-4bdd-b2be-9eaab784591f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:29:09.389404 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.389375    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3478799b-da94-4bdd-b2be-9eaab784591f-openshift-service-ca\") pod \"maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs\" (UID: \"3478799b-da94-4bdd-b2be-9eaab784591f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:29:09.389577 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.389429    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3478799b-da94-4bdd-b2be-9eaab784591f-maas-api-tls\") pod \"maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs\" (UID: \"3478799b-da94-4bdd-b2be-9eaab784591f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:29:09.389577 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.389448    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjc6b\" (UniqueName: \"kubernetes.io/projected/3478799b-da94-4bdd-b2be-9eaab784591f-kube-api-access-mjc6b\") pod \"maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs\" (UID: \"3478799b-da94-4bdd-b2be-9eaab784591f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:29:09.389577 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:29:09.389553    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-serving-cert: secret "maas-api-e2e-api-a-eb6c99-serving-cert" not found
Jul 30 04:29:09.389692 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:29:09.389628    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3478799b-da94-4bdd-b2be-9eaab784591f-maas-api-tls podName:3478799b-da94-4bdd-b2be-9eaab784591f nodeName:}" failed. No retries permitted until 2026-07-30 04:29:09.889609296 +0000 UTC m=+2648.928320293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3478799b-da94-4bdd-b2be-9eaab784591f-maas-api-tls") pod "maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs" (UID: "3478799b-da94-4bdd-b2be-9eaab784591f") : secret "maas-api-e2e-api-a-eb6c99-serving-cert" not found
Jul 30 04:29:09.390027 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.390008    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3478799b-da94-4bdd-b2be-9eaab784591f-openshift-service-ca\") pod \"maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs\" (UID: \"3478799b-da94-4bdd-b2be-9eaab784591f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:29:09.397663 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.397642    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjc6b\" (UniqueName: \"kubernetes.io/projected/3478799b-da94-4bdd-b2be-9eaab784591f-kube-api-access-mjc6b\") pod \"maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs\" (UID: \"3478799b-da94-4bdd-b2be-9eaab784591f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:29:09.682332 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.682294    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:09.777288 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.777248    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:09.817959 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.817929    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"]
Jul 30 04:29:09.821471 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.821456    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"
Jul 30 04:29:09.823683 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.823653    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-eb6c99\""
Jul 30 04:29:09.823811 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.823696    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-eb6c99-dockercfg-b9mps\""
Jul 30 04:29:09.831514 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.831495    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"]
Jul 30 04:29:09.848698 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.848673    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:09.893041 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.893011    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3478799b-da94-4bdd-b2be-9eaab784591f-maas-api-tls\") pod \"maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs\" (UID: \"3478799b-da94-4bdd-b2be-9eaab784591f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:29:09.893169 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.893060    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw8q4\" (UniqueName: \"kubernetes.io/projected/31592000-25af-47ca-8928-3f23612e2e2e-kube-api-access-lw8q4\") pod \"payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj\" (UID: \"31592000-25af-47ca-8928-3f23612e2e2e\") " pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"
Jul 30 04:29:09.893212 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.893198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31592000-25af-47ca-8928-3f23612e2e2e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj\" (UID: \"31592000-25af-47ca-8928-3f23612e2e2e\") " pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"
Jul 30 04:29:09.895494 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.895476    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3478799b-da94-4bdd-b2be-9eaab784591f-maas-api-tls\") pod \"maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs\" (UID: \"3478799b-da94-4bdd-b2be-9eaab784591f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:29:09.993939 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.993874    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw8q4\" (UniqueName: \"kubernetes.io/projected/31592000-25af-47ca-8928-3f23612e2e2e-kube-api-access-lw8q4\") pod \"payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj\" (UID: \"31592000-25af-47ca-8928-3f23612e2e2e\") " pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"
Jul 30 04:29:09.993939 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.993925    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31592000-25af-47ca-8928-3f23612e2e2e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj\" (UID: \"31592000-25af-47ca-8928-3f23612e2e2e\") " pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"
Jul 30 04:29:09.994579 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:09.994563    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31592000-25af-47ca-8928-3f23612e2e2e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj\" (UID: \"31592000-25af-47ca-8928-3f23612e2e2e\") " pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"
Jul 30 04:29:10.003111 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:10.003078    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw8q4\" (UniqueName: \"kubernetes.io/projected/31592000-25af-47ca-8928-3f23612e2e2e-kube-api-access-lw8q4\") pod \"payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj\" (UID: \"31592000-25af-47ca-8928-3f23612e2e2e\") " pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"
Jul 30 04:29:10.080402 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:10.080371    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:29:10.135030 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:10.133565    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"
Jul 30 04:29:10.226865 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:10.226836    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"]
Jul 30 04:29:10.229178 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:29:10.229147    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3478799b_da94_4bdd_b2be_9eaab784591f.slice/crio-eb6390e7db16a91ddeae2a9ae454b9a245a5f69e5727176da62394f133fc375f WatchSource:0}: Error finding container eb6390e7db16a91ddeae2a9ae454b9a245a5f69e5727176da62394f133fc375f: Status 404 returned error can't find the container with id eb6390e7db16a91ddeae2a9ae454b9a245a5f69e5727176da62394f133fc375f
Jul 30 04:29:10.230593 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:10.230576    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 04:29:10.274134 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:10.274112    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"]
Jul 30 04:29:10.276229 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:29:10.276205    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31592000_25af_47ca_8928_3f23612e2e2e.slice/crio-60e5f437eee33afd0329b357ebe715cbf063485d10c7e469da5e0caad208caf8 WatchSource:0}: Error finding container 60e5f437eee33afd0329b357ebe715cbf063485d10c7e469da5e0caad208caf8: Status 404 returned error can't find the container with id 60e5f437eee33afd0329b357ebe715cbf063485d10c7e469da5e0caad208caf8
Jul 30 04:29:10.469127 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:10.469083    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj" event={"ID":"31592000-25af-47ca-8928-3f23612e2e2e","Type":"ContainerStarted","Data":"1404e5942c88e6317cd0e9467de66c5c0eb3439b6fc8d5347786159ff8a02479"}
Jul 30 04:29:10.469306 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:10.469150    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"
Jul 30 04:29:10.469306 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:10.469173    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj" event={"ID":"31592000-25af-47ca-8928-3f23612e2e2e","Type":"ContainerStarted","Data":"60e5f437eee33afd0329b357ebe715cbf063485d10c7e469da5e0caad208caf8"}
Jul 30 04:29:10.470312 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:10.470289    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs" event={"ID":"3478799b-da94-4bdd-b2be-9eaab784591f","Type":"ContainerStarted","Data":"eb6390e7db16a91ddeae2a9ae454b9a245a5f69e5727176da62394f133fc375f"}
Jul 30 04:29:10.485814 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:10.485769    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj" podStartSLOduration=1.485755852 podStartE2EDuration="1.485755852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:29:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:29:10.484710453 +0000 UTC m=+2649.523421467" watchObservedRunningTime="2026-07-30 04:29:10.485755852 +0000 UTC m=+2649.524466874"
Jul 30 04:29:11.478230 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:11.478191    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs" event={"ID":"3478799b-da94-4bdd-b2be-9eaab784591f","Type":"ContainerStarted","Data":"dad902e0d779f2e06bf3e1e667b041b97f9c05eb68500421331cbbd3018f7a8d"}
Jul 30 04:29:11.478640 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:11.478261    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:29:11.493293 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:11.493244    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs" podStartSLOduration=2.022116798 podStartE2EDuration="2.49322401s" totalImagesPullingTime="471.107212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:29:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:29:11.492653667 +0000 UTC m=+2650.531364686" watchObservedRunningTime="2026-07-30 04:29:11.49322401 +0000 UTC m=+2650.531935028"
Jul 30 04:29:17.488848 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:17.488821    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:29:21.479987 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:21.479955    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"
Jul 30 04:29:33.051659 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:33.051623    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:34.175754 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:34.175720    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:34.282123 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:34.282089    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:37.326853 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:37.326810    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:37.427899 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:37.427867    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:38.421797 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.421765    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"]
Jul 30 04:29:38.425187 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.425165    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:29:38.427516 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.427493    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-587da0-serving-cert\""
Jul 30 04:29:38.435912 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.435880    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"]
Jul 30 04:29:38.445784 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.445761    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwfkq\" (UniqueName: \"kubernetes.io/projected/85d4c1bf-64aa-41b5-ae18-22ea398ece54-kube-api-access-cwfkq\") pod \"maas-api-e2e-api-b-587da0-765b77c68d-jkpgt\" (UID: \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:29:38.445895 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.445814    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85d4c1bf-64aa-41b5-ae18-22ea398ece54-maas-api-tls\") pod \"maas-api-e2e-api-b-587da0-765b77c68d-jkpgt\" (UID: \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:29:38.445959 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.445928    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85d4c1bf-64aa-41b5-ae18-22ea398ece54-openshift-service-ca\") pod \"maas-api-e2e-api-b-587da0-765b77c68d-jkpgt\" (UID: \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:29:38.461209 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.461184    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:38.546295 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.546271    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85d4c1bf-64aa-41b5-ae18-22ea398ece54-openshift-service-ca\") pod \"maas-api-e2e-api-b-587da0-765b77c68d-jkpgt\" (UID: \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:29:38.546408 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.546361    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwfkq\" (UniqueName: \"kubernetes.io/projected/85d4c1bf-64aa-41b5-ae18-22ea398ece54-kube-api-access-cwfkq\") pod \"maas-api-e2e-api-b-587da0-765b77c68d-jkpgt\" (UID: \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:29:38.546482 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.546409    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85d4c1bf-64aa-41b5-ae18-22ea398ece54-maas-api-tls\") pod \"maas-api-e2e-api-b-587da0-765b77c68d-jkpgt\" (UID: \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:29:38.546807 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:29:38.546778    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-serving-cert: secret "maas-api-e2e-api-b-587da0-serving-cert" not found
Jul 30 04:29:38.546916 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:29:38.546860    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85d4c1bf-64aa-41b5-ae18-22ea398ece54-maas-api-tls podName:85d4c1bf-64aa-41b5-ae18-22ea398ece54 nodeName:}" failed. No retries permitted until 2026-07-30 04:29:39.046839461 +0000 UTC m=+2678.085550480 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/85d4c1bf-64aa-41b5-ae18-22ea398ece54-maas-api-tls") pod "maas-api-e2e-api-b-587da0-765b77c68d-jkpgt" (UID: "85d4c1bf-64aa-41b5-ae18-22ea398ece54") : secret "maas-api-e2e-api-b-587da0-serving-cert" not found
Jul 30 04:29:38.547055 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.547029    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85d4c1bf-64aa-41b5-ae18-22ea398ece54-openshift-service-ca\") pod \"maas-api-e2e-api-b-587da0-765b77c68d-jkpgt\" (UID: \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:29:38.554924 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.554896    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwfkq\" (UniqueName: \"kubernetes.io/projected/85d4c1bf-64aa-41b5-ae18-22ea398ece54-kube-api-access-cwfkq\") pod \"maas-api-e2e-api-b-587da0-765b77c68d-jkpgt\" (UID: \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:29:38.630971 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.630949    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:38.948272 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:38.948231    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:39.046549 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.046504    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:29:39.049705 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.049686    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85d4c1bf-64aa-41b5-ae18-22ea398ece54-maas-api-tls\") pod \"maas-api-e2e-api-b-587da0-765b77c68d-jkpgt\" (UID: \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:29:39.052198 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.052170    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85d4c1bf-64aa-41b5-ae18-22ea398ece54-maas-api-tls\") pod \"maas-api-e2e-api-b-587da0-765b77c68d-jkpgt\" (UID: \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:29:39.129812 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.129773    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"]
Jul 30 04:29:39.132922 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.132907    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"
Jul 30 04:29:39.135112 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.135089    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-587da0\""
Jul 30 04:29:39.135234 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.135152    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-587da0-dockercfg-qfgq8\""
Jul 30 04:29:39.145560 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.144489    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"]
Jul 30 04:29:39.151371 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.151339    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4j4n\" (UniqueName: \"kubernetes.io/projected/5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc-kube-api-access-g4j4n\") pod \"payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5\" (UID: \"5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"
Jul 30 04:29:39.151502 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.151440    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5\" (UID: \"5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"
Jul 30 04:29:39.252141 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.252074    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4j4n\" (UniqueName: \"kubernetes.io/projected/5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc-kube-api-access-g4j4n\") pod \"payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5\" (UID: \"5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"
Jul 30 04:29:39.252141 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.252130    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5\" (UID: \"5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"
Jul 30 04:29:39.252857 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.252838    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5\" (UID: \"5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"
Jul 30 04:29:39.259655 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.259633    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4j4n\" (UniqueName: \"kubernetes.io/projected/5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc-kube-api-access-g4j4n\") pod \"payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5\" (UID: \"5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"
Jul 30 04:29:39.339400 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.339366    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:29:39.454801 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.454771    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"
Jul 30 04:29:39.472743 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.472718    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"]
Jul 30 04:29:39.474867 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:29:39.474841    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d4c1bf_64aa_41b5_ae18_22ea398ece54.slice/crio-9243d051f1756455b3aaaa9432a91aa49d1f545e72029de0a1b0ec08edcd99c6 WatchSource:0}: Error finding container 9243d051f1756455b3aaaa9432a91aa49d1f545e72029de0a1b0ec08edcd99c6: Status 404 returned error can't find the container with id 9243d051f1756455b3aaaa9432a91aa49d1f545e72029de0a1b0ec08edcd99c6
Jul 30 04:29:39.589716 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.589689    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"]
Jul 30 04:29:39.591322 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:29:39.591295    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5af4b9a5_c7d7_4bfe_a66f_48ad201df2fc.slice/crio-973bf4d0aab587376fa16ca28f17d37827d93095ef87e3b56cc00bb53466976a WatchSource:0}: Error finding container 973bf4d0aab587376fa16ca28f17d37827d93095ef87e3b56cc00bb53466976a: Status 404 returned error can't find the container with id 973bf4d0aab587376fa16ca28f17d37827d93095ef87e3b56cc00bb53466976a
Jul 30 04:29:39.616820 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.616794    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5" event={"ID":"5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc","Type":"ContainerStarted","Data":"973bf4d0aab587376fa16ca28f17d37827d93095ef87e3b56cc00bb53466976a"}
Jul 30 04:29:39.617909 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:39.617879    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt" event={"ID":"85d4c1bf-64aa-41b5-ae18-22ea398ece54","Type":"ContainerStarted","Data":"9243d051f1756455b3aaaa9432a91aa49d1f545e72029de0a1b0ec08edcd99c6"}
Jul 30 04:29:40.624189 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:40.624156    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt" event={"ID":"85d4c1bf-64aa-41b5-ae18-22ea398ece54","Type":"ContainerStarted","Data":"f575e4348dc6170f66d36668c8e58060ba2d5c9c29977a147ee4d8c18700b8be"}
Jul 30 04:29:40.624656 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:40.624223    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:29:40.625805 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:40.625776    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5" event={"ID":"5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc","Type":"ContainerStarted","Data":"ad019e7608c5755346cf508ab33ece0b17d46518842e3d1435c4b4798301659c"}
Jul 30 04:29:40.625934 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:40.625895    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"
Jul 30 04:29:40.640661 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:40.640618    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt" podStartSLOduration=2.268277185 podStartE2EDuration="2.640605932s" totalImagesPullingTime="372.328747ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:29:40.63811339 +0000 UTC m=+2679.676824409" watchObservedRunningTime="2026-07-30 04:29:40.640605932 +0000 UTC m=+2679.679316982"
Jul 30 04:29:40.652949 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:40.652904    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5" podStartSLOduration=1.65288941 podStartE2EDuration="1.65288941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:29:40.651986177 +0000 UTC m=+2679.690697195" watchObservedRunningTime="2026-07-30 04:29:40.65288941 +0000 UTC m=+2679.691600428"
Jul 30 04:29:46.637894 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:46.637815    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:29:51.634184 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:29:51.634151    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"
Jul 30 04:30:01.644455 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:01.644422    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:30:01.652947 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:01.652926    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:30:02.275717 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:02.275680    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:20.805651 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:20.805613    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"]
Jul 30 04:30:20.806256 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:20.805880    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs" podUID="3478799b-da94-4bdd-b2be-9eaab784591f" containerName="maas-api" containerID="cri-o://dad902e0d779f2e06bf3e1e667b041b97f9c05eb68500421331cbbd3018f7a8d" gracePeriod=30
Jul 30 04:30:20.842879 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:20.842852    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:20.872938 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:20.872909    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"]
Jul 30 04:30:20.873168 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:20.873141    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj" podUID="31592000-25af-47ca-8928-3f23612e2e2e" containerName="payload-processing" containerID="cri-o://1404e5942c88e6317cd0e9467de66c5c0eb3439b6fc8d5347786159ff8a02479" gracePeriod=30
Jul 30 04:30:20.941889 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:20.941859    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:21.020247 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.020192    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:21.080620 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.080512    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:21.131321 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.131286    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:30:21.160473 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.160443    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"
Jul 30 04:30:21.290330 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.290302    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjc6b\" (UniqueName: \"kubernetes.io/projected/3478799b-da94-4bdd-b2be-9eaab784591f-kube-api-access-mjc6b\") pod \"3478799b-da94-4bdd-b2be-9eaab784591f\" (UID: \"3478799b-da94-4bdd-b2be-9eaab784591f\") "
Jul 30 04:30:21.290472 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.290351    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3478799b-da94-4bdd-b2be-9eaab784591f-openshift-service-ca\") pod \"3478799b-da94-4bdd-b2be-9eaab784591f\" (UID: \"3478799b-da94-4bdd-b2be-9eaab784591f\") "
Jul 30 04:30:21.290472 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.290396    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lw8q4\" (UniqueName: \"kubernetes.io/projected/31592000-25af-47ca-8928-3f23612e2e2e-kube-api-access-lw8q4\") pod \"31592000-25af-47ca-8928-3f23612e2e2e\" (UID: \"31592000-25af-47ca-8928-3f23612e2e2e\") "
Jul 30 04:30:21.290472 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.290414    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31592000-25af-47ca-8928-3f23612e2e2e-plugins-config-volume\") pod \"31592000-25af-47ca-8928-3f23612e2e2e\" (UID: \"31592000-25af-47ca-8928-3f23612e2e2e\") "
Jul 30 04:30:21.290472 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.290437    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3478799b-da94-4bdd-b2be-9eaab784591f-maas-api-tls\") pod \"3478799b-da94-4bdd-b2be-9eaab784591f\" (UID: \"3478799b-da94-4bdd-b2be-9eaab784591f\") "
Jul 30 04:30:21.290864 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.290835    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3478799b-da94-4bdd-b2be-9eaab784591f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3478799b-da94-4bdd-b2be-9eaab784591f" (UID: "3478799b-da94-4bdd-b2be-9eaab784591f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:30:21.290864 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.290847    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31592000-25af-47ca-8928-3f23612e2e2e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "31592000-25af-47ca-8928-3f23612e2e2e" (UID: "31592000-25af-47ca-8928-3f23612e2e2e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:30:21.292734 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.292707    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31592000-25af-47ca-8928-3f23612e2e2e-kube-api-access-lw8q4" (OuterVolumeSpecName: "kube-api-access-lw8q4") pod "31592000-25af-47ca-8928-3f23612e2e2e" (UID: "31592000-25af-47ca-8928-3f23612e2e2e"). InnerVolumeSpecName "kube-api-access-lw8q4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:30:21.292829 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.292756    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3478799b-da94-4bdd-b2be-9eaab784591f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3478799b-da94-4bdd-b2be-9eaab784591f" (UID: "3478799b-da94-4bdd-b2be-9eaab784591f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:30:21.292829 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.292772    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3478799b-da94-4bdd-b2be-9eaab784591f-kube-api-access-mjc6b" (OuterVolumeSpecName: "kube-api-access-mjc6b") pod "3478799b-da94-4bdd-b2be-9eaab784591f" (UID: "3478799b-da94-4bdd-b2be-9eaab784591f"). InnerVolumeSpecName "kube-api-access-mjc6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:30:21.391021 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.390936    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lw8q4\" (UniqueName: \"kubernetes.io/projected/31592000-25af-47ca-8928-3f23612e2e2e-kube-api-access-lw8q4\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:30:21.391021 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.390969    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31592000-25af-47ca-8928-3f23612e2e2e-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:30:21.391021 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.390980    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3478799b-da94-4bdd-b2be-9eaab784591f-maas-api-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:30:21.391021 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.390990    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjc6b\" (UniqueName: \"kubernetes.io/projected/3478799b-da94-4bdd-b2be-9eaab784591f-kube-api-access-mjc6b\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:30:21.391021 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.391000    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3478799b-da94-4bdd-b2be-9eaab784591f-openshift-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:30:21.392619 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.392594    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:21.832592 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.832555    2569 generic.go:356] "Generic (PLEG): container finished" podID="3478799b-da94-4bdd-b2be-9eaab784591f" containerID="dad902e0d779f2e06bf3e1e667b041b97f9c05eb68500421331cbbd3018f7a8d" exitCode=0
Jul 30 04:30:21.832592 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.832575    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs" event={"ID":"3478799b-da94-4bdd-b2be-9eaab784591f","Type":"ContainerDied","Data":"dad902e0d779f2e06bf3e1e667b041b97f9c05eb68500421331cbbd3018f7a8d"}
Jul 30 04:30:21.833049 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.832614    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"
Jul 30 04:30:21.833049 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.832642    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs" event={"ID":"3478799b-da94-4bdd-b2be-9eaab784591f","Type":"ContainerDied","Data":"eb6390e7db16a91ddeae2a9ae454b9a245a5f69e5727176da62394f133fc375f"}
Jul 30 04:30:21.833049 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.832674    2569 scope.go:117] "RemoveContainer" containerID="dad902e0d779f2e06bf3e1e667b041b97f9c05eb68500421331cbbd3018f7a8d"
Jul 30 04:30:21.834309 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.834290    2569 generic.go:356] "Generic (PLEG): container finished" podID="31592000-25af-47ca-8928-3f23612e2e2e" containerID="1404e5942c88e6317cd0e9467de66c5c0eb3439b6fc8d5347786159ff8a02479" exitCode=0
Jul 30 04:30:21.834413 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.834320    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj" event={"ID":"31592000-25af-47ca-8928-3f23612e2e2e","Type":"ContainerDied","Data":"1404e5942c88e6317cd0e9467de66c5c0eb3439b6fc8d5347786159ff8a02479"}
Jul 30 04:30:21.834413 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.834356    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"
Jul 30 04:30:21.834413 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.834364    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj" event={"ID":"31592000-25af-47ca-8928-3f23612e2e2e","Type":"ContainerDied","Data":"60e5f437eee33afd0329b357ebe715cbf063485d10c7e469da5e0caad208caf8"}
Jul 30 04:30:21.842264 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.842246    2569 scope.go:117] "RemoveContainer" containerID="dad902e0d779f2e06bf3e1e667b041b97f9c05eb68500421331cbbd3018f7a8d"
Jul 30 04:30:21.842500 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:30:21.842484    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dad902e0d779f2e06bf3e1e667b041b97f9c05eb68500421331cbbd3018f7a8d\": container with ID starting with dad902e0d779f2e06bf3e1e667b041b97f9c05eb68500421331cbbd3018f7a8d not found: ID does not exist" containerID="dad902e0d779f2e06bf3e1e667b041b97f9c05eb68500421331cbbd3018f7a8d"
Jul 30 04:30:21.842571 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.842506    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dad902e0d779f2e06bf3e1e667b041b97f9c05eb68500421331cbbd3018f7a8d"} err="failed to get container status \"dad902e0d779f2e06bf3e1e667b041b97f9c05eb68500421331cbbd3018f7a8d\": rpc error: code = NotFound desc = could not find container \"dad902e0d779f2e06bf3e1e667b041b97f9c05eb68500421331cbbd3018f7a8d\": container with ID starting with dad902e0d779f2e06bf3e1e667b041b97f9c05eb68500421331cbbd3018f7a8d not found: ID does not exist"
Jul 30 04:30:21.842571 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.842522    2569 scope.go:117] "RemoveContainer" containerID="1404e5942c88e6317cd0e9467de66c5c0eb3439b6fc8d5347786159ff8a02479"
Jul 30 04:30:21.850593 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.850566    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"]
Jul 30 04:30:21.851260 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.851238    2569 scope.go:117] "RemoveContainer" containerID="1404e5942c88e6317cd0e9467de66c5c0eb3439b6fc8d5347786159ff8a02479"
Jul 30 04:30:21.851509 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:30:21.851488    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1404e5942c88e6317cd0e9467de66c5c0eb3439b6fc8d5347786159ff8a02479\": container with ID starting with 1404e5942c88e6317cd0e9467de66c5c0eb3439b6fc8d5347786159ff8a02479 not found: ID does not exist" containerID="1404e5942c88e6317cd0e9467de66c5c0eb3439b6fc8d5347786159ff8a02479"
Jul 30 04:30:21.851608 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.851518    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1404e5942c88e6317cd0e9467de66c5c0eb3439b6fc8d5347786159ff8a02479"} err="failed to get container status \"1404e5942c88e6317cd0e9467de66c5c0eb3439b6fc8d5347786159ff8a02479\": rpc error: code = NotFound desc = could not find container \"1404e5942c88e6317cd0e9467de66c5c0eb3439b6fc8d5347786159ff8a02479\": container with ID starting with 1404e5942c88e6317cd0e9467de66c5c0eb3439b6fc8d5347786159ff8a02479 not found: ID does not exist"
Jul 30 04:30:21.852996 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.852978    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eb6c99-694f85d67d-djsgs"]
Jul 30 04:30:21.860245 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.860224    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"]
Jul 30 04:30:21.863813 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:21.863793    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eb6c99-54b9546564-wrhtj"]
Jul 30 04:30:23.464924 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:23.464888    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31592000-25af-47ca-8928-3f23612e2e2e" path="/var/lib/kubelet/pods/31592000-25af-47ca-8928-3f23612e2e2e/volumes"
Jul 30 04:30:23.465298 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:23.465245    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3478799b-da94-4bdd-b2be-9eaab784591f" path="/var/lib/kubelet/pods/3478799b-da94-4bdd-b2be-9eaab784591f/volumes"
Jul 30 04:30:26.949273 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:26.949237    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:27.011574 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:27.011527    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:27.228064 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:27.227962    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:27.449918 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:27.449886    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:43.142416 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.142382    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"]
Jul 30 04:30:43.142907 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.142614    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt" podUID="85d4c1bf-64aa-41b5-ae18-22ea398ece54" containerName="maas-api" containerID="cri-o://f575e4348dc6170f66d36668c8e58060ba2d5c9c29977a147ee4d8c18700b8be" gracePeriod=30
Jul 30 04:30:43.174371 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.174343    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:43.207784 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.207755    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"]
Jul 30 04:30:43.208087 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.207979    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5" podUID="5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc" containerName="payload-pre-processing" containerID="cri-o://ad019e7608c5755346cf508ab33ece0b17d46518842e3d1435c4b4798301659c" gracePeriod=30
Jul 30 04:30:43.240190 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.240156    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:43.341591 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.341558    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:43.403332 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.403248    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:43.457557 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.457518    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:30:43.465291 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.465270    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85d4c1bf-64aa-41b5-ae18-22ea398ece54-openshift-service-ca\") pod \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\" (UID: \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\") "
Jul 30 04:30:43.465391 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.465327    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85d4c1bf-64aa-41b5-ae18-22ea398ece54-maas-api-tls\") pod \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\" (UID: \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\") "
Jul 30 04:30:43.465459 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.465440    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwfkq\" (UniqueName: \"kubernetes.io/projected/85d4c1bf-64aa-41b5-ae18-22ea398ece54-kube-api-access-cwfkq\") pod \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\" (UID: \"85d4c1bf-64aa-41b5-ae18-22ea398ece54\") "
Jul 30 04:30:43.465698 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.465667    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85d4c1bf-64aa-41b5-ae18-22ea398ece54-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "85d4c1bf-64aa-41b5-ae18-22ea398ece54" (UID: "85d4c1bf-64aa-41b5-ae18-22ea398ece54"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:30:43.467575 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.467551    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85d4c1bf-64aa-41b5-ae18-22ea398ece54-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "85d4c1bf-64aa-41b5-ae18-22ea398ece54" (UID: "85d4c1bf-64aa-41b5-ae18-22ea398ece54"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:30:43.467729 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.467709    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85d4c1bf-64aa-41b5-ae18-22ea398ece54-kube-api-access-cwfkq" (OuterVolumeSpecName: "kube-api-access-cwfkq") pod "85d4c1bf-64aa-41b5-ae18-22ea398ece54" (UID: "85d4c1bf-64aa-41b5-ae18-22ea398ece54"). InnerVolumeSpecName "kube-api-access-cwfkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:30:43.484224 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.484207    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"
Jul 30 04:30:43.566759 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.566733    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc-plugins-config-volume\") pod \"5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc\" (UID: \"5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc\") "
Jul 30 04:30:43.566878 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.566767    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4j4n\" (UniqueName: \"kubernetes.io/projected/5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc-kube-api-access-g4j4n\") pod \"5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc\" (UID: \"5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc\") "
Jul 30 04:30:43.567067 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.567050    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85d4c1bf-64aa-41b5-ae18-22ea398ece54-openshift-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:30:43.567117 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.567076    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85d4c1bf-64aa-41b5-ae18-22ea398ece54-maas-api-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:30:43.567117 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.567093    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwfkq\" (UniqueName: \"kubernetes.io/projected/85d4c1bf-64aa-41b5-ae18-22ea398ece54-kube-api-access-cwfkq\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:30:43.567188 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.567133    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc" (UID: "5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:30:43.568912 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.568892    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc-kube-api-access-g4j4n" (OuterVolumeSpecName: "kube-api-access-g4j4n") pod "5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc" (UID: "5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc"). InnerVolumeSpecName "kube-api-access-g4j4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:30:43.668436 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.668364    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:30:43.668436 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.668388    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4j4n\" (UniqueName: \"kubernetes.io/projected/5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc-kube-api-access-g4j4n\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:30:43.722774 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.722742    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:43.947927 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.947830    2569 generic.go:356] "Generic (PLEG): container finished" podID="5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc" containerID="ad019e7608c5755346cf508ab33ece0b17d46518842e3d1435c4b4798301659c" exitCode=0
Jul 30 04:30:43.947927 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.947878    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"
Jul 30 04:30:43.948137 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.947928    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5" event={"ID":"5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc","Type":"ContainerDied","Data":"ad019e7608c5755346cf508ab33ece0b17d46518842e3d1435c4b4798301659c"}
Jul 30 04:30:43.948137 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.947980    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5" event={"ID":"5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc","Type":"ContainerDied","Data":"973bf4d0aab587376fa16ca28f17d37827d93095ef87e3b56cc00bb53466976a"}
Jul 30 04:30:43.948137 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.948016    2569 scope.go:117] "RemoveContainer" containerID="ad019e7608c5755346cf508ab33ece0b17d46518842e3d1435c4b4798301659c"
Jul 30 04:30:43.949313 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.949287    2569 generic.go:356] "Generic (PLEG): container finished" podID="85d4c1bf-64aa-41b5-ae18-22ea398ece54" containerID="f575e4348dc6170f66d36668c8e58060ba2d5c9c29977a147ee4d8c18700b8be" exitCode=0
Jul 30 04:30:43.949438 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.949322    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt" event={"ID":"85d4c1bf-64aa-41b5-ae18-22ea398ece54","Type":"ContainerDied","Data":"f575e4348dc6170f66d36668c8e58060ba2d5c9c29977a147ee4d8c18700b8be"}
Jul 30 04:30:43.949438 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.949352    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt" event={"ID":"85d4c1bf-64aa-41b5-ae18-22ea398ece54","Type":"ContainerDied","Data":"9243d051f1756455b3aaaa9432a91aa49d1f545e72029de0a1b0ec08edcd99c6"}
Jul 30 04:30:43.949438 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.949395    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"
Jul 30 04:30:43.959043 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.959025    2569 scope.go:117] "RemoveContainer" containerID="ad019e7608c5755346cf508ab33ece0b17d46518842e3d1435c4b4798301659c"
Jul 30 04:30:43.959302 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:30:43.959283    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad019e7608c5755346cf508ab33ece0b17d46518842e3d1435c4b4798301659c\": container with ID starting with ad019e7608c5755346cf508ab33ece0b17d46518842e3d1435c4b4798301659c not found: ID does not exist" containerID="ad019e7608c5755346cf508ab33ece0b17d46518842e3d1435c4b4798301659c"
Jul 30 04:30:43.959381 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.959314    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad019e7608c5755346cf508ab33ece0b17d46518842e3d1435c4b4798301659c"} err="failed to get container status \"ad019e7608c5755346cf508ab33ece0b17d46518842e3d1435c4b4798301659c\": rpc error: code = NotFound desc = could not find container \"ad019e7608c5755346cf508ab33ece0b17d46518842e3d1435c4b4798301659c\": container with ID starting with ad019e7608c5755346cf508ab33ece0b17d46518842e3d1435c4b4798301659c not found: ID does not exist"
Jul 30 04:30:43.959381 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.959338    2569 scope.go:117] "RemoveContainer" containerID="f575e4348dc6170f66d36668c8e58060ba2d5c9c29977a147ee4d8c18700b8be"
Jul 30 04:30:43.967724 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.967703    2569 scope.go:117] "RemoveContainer" containerID="f575e4348dc6170f66d36668c8e58060ba2d5c9c29977a147ee4d8c18700b8be"
Jul 30 04:30:43.968100 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:30:43.968033    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f575e4348dc6170f66d36668c8e58060ba2d5c9c29977a147ee4d8c18700b8be\": container with ID starting with f575e4348dc6170f66d36668c8e58060ba2d5c9c29977a147ee4d8c18700b8be not found: ID does not exist" containerID="f575e4348dc6170f66d36668c8e58060ba2d5c9c29977a147ee4d8c18700b8be"
Jul 30 04:30:43.968100 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.968068    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f575e4348dc6170f66d36668c8e58060ba2d5c9c29977a147ee4d8c18700b8be"} err="failed to get container status \"f575e4348dc6170f66d36668c8e58060ba2d5c9c29977a147ee4d8c18700b8be\": rpc error: code = NotFound desc = could not find container \"f575e4348dc6170f66d36668c8e58060ba2d5c9c29977a147ee4d8c18700b8be\": container with ID starting with f575e4348dc6170f66d36668c8e58060ba2d5c9c29977a147ee4d8c18700b8be not found: ID does not exist"
Jul 30 04:30:43.970087 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.970069    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"]
Jul 30 04:30:43.973607 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.973586    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-587da0-7c77cd5dd8-x7jd5"]
Jul 30 04:30:43.982999 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.982979    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"]
Jul 30 04:30:43.986661 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:43.986642    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-587da0-765b77c68d-jkpgt"]
Jul 30 04:30:45.465464 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:45.465429    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc" path="/var/lib/kubelet/pods/5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc/volumes"
Jul 30 04:30:45.465882 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:45.465865    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85d4c1bf-64aa-41b5-ae18-22ea398ece54" path="/var/lib/kubelet/pods/85d4c1bf-64aa-41b5-ae18-22ea398ece54/volumes"
Jul 30 04:30:49.474824 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:49.474790    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:49.537920 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:49.537889    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:49.747272 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:49.747189    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:49.986202 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:49.986171    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:50.322258 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:50.322224    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:50.632153 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:50.632077    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:50.723434 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:50.723403    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:53.666604 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:53.666512    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:53.755977 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:53.755913    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:54.752677 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.752647    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"]
Jul 30 04:30:54.753163 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.753012    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3478799b-da94-4bdd-b2be-9eaab784591f" containerName="maas-api"
Jul 30 04:30:54.753163 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.753027    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3478799b-da94-4bdd-b2be-9eaab784591f" containerName="maas-api"
Jul 30 04:30:54.753163 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.753041    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31592000-25af-47ca-8928-3f23612e2e2e" containerName="payload-processing"
Jul 30 04:30:54.753163 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.753048    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="31592000-25af-47ca-8928-3f23612e2e2e" containerName="payload-processing"
Jul 30 04:30:54.753163 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.753057    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85d4c1bf-64aa-41b5-ae18-22ea398ece54" containerName="maas-api"
Jul 30 04:30:54.753163 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.753065    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="85d4c1bf-64aa-41b5-ae18-22ea398ece54" containerName="maas-api"
Jul 30 04:30:54.753163 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.753076    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc" containerName="payload-pre-processing"
Jul 30 04:30:54.753163 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.753085    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc" containerName="payload-pre-processing"
Jul 30 04:30:54.753601 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.753225    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="85d4c1bf-64aa-41b5-ae18-22ea398ece54" containerName="maas-api"
Jul 30 04:30:54.753601 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.753238    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3478799b-da94-4bdd-b2be-9eaab784591f" containerName="maas-api"
Jul 30 04:30:54.753601 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.753249    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5af4b9a5-c7d7-4bfe-a66f-48ad201df2fc" containerName="payload-pre-processing"
Jul 30 04:30:54.753601 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.753258    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="31592000-25af-47ca-8928-3f23612e2e2e" containerName="payload-processing"
Jul 30 04:30:54.757709 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.757690    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:30:54.760143 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.760123    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-324a29-serving-cert\""
Jul 30 04:30:54.760388 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.760373    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lzvw2\""
Jul 30 04:30:54.766431 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.766408    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"]
Jul 30 04:30:54.784175 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.784157    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:54.854462 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.854440    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-maas-api-tls\") pod \"maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8\" (UID: \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:30:54.854594 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.854483    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8\" (UID: \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:30:54.854639 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.854600    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsjx6\" (UniqueName: \"kubernetes.io/projected/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-kube-api-access-bsjx6\") pod \"maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8\" (UID: \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:30:54.955683 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.955653    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsjx6\" (UniqueName: \"kubernetes.io/projected/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-kube-api-access-bsjx6\") pod \"maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8\" (UID: \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:30:54.955803 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.955708    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-maas-api-tls\") pod \"maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8\" (UID: \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:30:54.955803 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.955739    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8\" (UID: \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:30:54.955889 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:30:54.955869    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-serving-cert: secret "maas-api-e2e-shared-a-324a29-serving-cert" not found
Jul 30 04:30:54.955958 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:30:54.955947    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-maas-api-tls podName:e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5 nodeName:}" failed. No retries permitted until 2026-07-30 04:30:55.455928067 +0000 UTC m=+2754.494639065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-maas-api-tls") pod "maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8" (UID: "e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5") : secret "maas-api-e2e-shared-a-324a29-serving-cert" not found
Jul 30 04:30:54.956253 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.956239    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8\" (UID: \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:30:54.963902 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:54.963873    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsjx6\" (UniqueName: \"kubernetes.io/projected/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-kube-api-access-bsjx6\") pod \"maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8\" (UID: \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:30:55.057087 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:55.057058    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:55.303288 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:55.303261    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:55.399007 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:55.398931    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:55.459715 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:55.459691    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-maas-api-tls\") pod \"maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8\" (UID: \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:30:55.462619 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:55.462589    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-maas-api-tls\") pod \"maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8\" (UID: \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:30:55.670158 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:55.670086    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:30:55.804947 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:55.804922    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"]
Jul 30 04:30:55.806994 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:30:55.806966    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1ebbc43_e61d_4032_a45f_1f4a9a3e90c5.slice/crio-d291e38a003f35dcbdc14402aea8f81e225251007ba8e31412b291e7aef826e2 WatchSource:0}: Error finding container d291e38a003f35dcbdc14402aea8f81e225251007ba8e31412b291e7aef826e2: Status 404 returned error can't find the container with id d291e38a003f35dcbdc14402aea8f81e225251007ba8e31412b291e7aef826e2
Jul 30 04:30:56.015916 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:56.015886    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8" event={"ID":"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5","Type":"ContainerStarted","Data":"d291e38a003f35dcbdc14402aea8f81e225251007ba8e31412b291e7aef826e2"}
Jul 30 04:30:57.022960 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:57.022920    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8" event={"ID":"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5","Type":"ContainerStarted","Data":"0502cbffa4493dd74cc2df5ba8b8169444d6851ffe8af5a48c334736ab1f61ff"}
Jul 30 04:30:57.023372 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:57.023090    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:30:57.039494 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:30:57.039441    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8" podStartSLOduration=2.571033172 podStartE2EDuration="3.039427151s" totalImagesPullingTime="468.393979ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:30:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:30:57.037284108 +0000 UTC m=+2756.075995127" watchObservedRunningTime="2026-07-30 04:30:57.039427151 +0000 UTC m=+2756.078138171"
Jul 30 04:31:03.032914 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:03.032871    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:31:09.875435 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:09.875393    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:18.676161 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:18.676123    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:19.035282 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:19.035240    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:19.349598 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:19.349492    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:19.450879 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:19.450844    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:21.857434 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:21.857330    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:21.953376 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:21.953345    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:23.138694 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.138645    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"]
Jul 30 04:31:23.142031 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.142015    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:31:23.144746 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.144727    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-64af9f-serving-cert\""
Jul 30 04:31:23.172183 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.172163    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"]
Jul 30 04:31:23.195320 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.195293    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:23.276274 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.276234    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rjlw\" (UniqueName: \"kubernetes.io/projected/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-kube-api-access-7rjlw\") pod \"maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7\" (UID: \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:31:23.276432 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.276334    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-maas-api-tls\") pod \"maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7\" (UID: \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:31:23.276432 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.276389    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-openshift-service-ca\") pod \"maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7\" (UID: \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:31:23.377523 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.377488    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-openshift-service-ca\") pod \"maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7\" (UID: \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:31:23.377708 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.377557    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rjlw\" (UniqueName: \"kubernetes.io/projected/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-kube-api-access-7rjlw\") pod \"maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7\" (UID: \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:31:23.377708 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.377604    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-maas-api-tls\") pod \"maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7\" (UID: \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:31:23.377795 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:31:23.377711    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-serving-cert: secret "maas-api-e2e-shared-b-64af9f-serving-cert" not found
Jul 30 04:31:23.377795 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:31:23.377767    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-maas-api-tls podName:7cc53b5a-d515-4c21-b7bd-fa28d8c08f84 nodeName:}" failed. No retries permitted until 2026-07-30 04:31:23.877752266 +0000 UTC m=+2782.916463264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-maas-api-tls") pod "maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7" (UID: "7cc53b5a-d515-4c21-b7bd-fa28d8c08f84") : secret "maas-api-e2e-shared-b-64af9f-serving-cert" not found
Jul 30 04:31:23.378134 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.378114    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-openshift-service-ca\") pod \"maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7\" (UID: \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:31:23.386395 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.386365    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rjlw\" (UniqueName: \"kubernetes.io/projected/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-kube-api-access-7rjlw\") pod \"maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7\" (UID: \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:31:23.669440 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.669405    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:23.766808 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.766775    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:23.791461 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.791434    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"]
Jul 30 04:31:23.794996 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.794981    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"
Jul 30 04:31:23.797152 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.797127    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-64af9f\""
Jul 30 04:31:23.797152 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.797151    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-64af9f-dockercfg-946d8\""
Jul 30 04:31:23.802232 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.802211    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"]
Jul 30 04:31:23.817359 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.817336    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"]
Jul 30 04:31:23.824435 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.824408    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"
Jul 30 04:31:23.830972 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.830950    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"]
Jul 30 04:31:23.841227 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.841194    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:23.883262 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.883234    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-maas-api-tls\") pod \"maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7\" (UID: \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:31:23.883389 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.883296    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd46d82e-52b7-4b2e-882d-edbc97582a1d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj\" (UID: \"bd46d82e-52b7-4b2e-882d-edbc97582a1d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"
Jul 30 04:31:23.883389 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.883336    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8np4\" (UniqueName: \"kubernetes.io/projected/bd46d82e-52b7-4b2e-882d-edbc97582a1d-kube-api-access-w8np4\") pod \"payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj\" (UID: \"bd46d82e-52b7-4b2e-882d-edbc97582a1d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"
Jul 30 04:31:23.885765 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.885744    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-maas-api-tls\") pod \"maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7\" (UID: \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:31:23.983989 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.983901    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecc7bb6a-6fa6-4237-84f7-030136d77069-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q\" (UID: \"ecc7bb6a-6fa6-4237-84f7-030136d77069\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"
Jul 30 04:31:23.983989 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.983953    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xqnt\" (UniqueName: \"kubernetes.io/projected/ecc7bb6a-6fa6-4237-84f7-030136d77069-kube-api-access-9xqnt\") pod \"payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q\" (UID: \"ecc7bb6a-6fa6-4237-84f7-030136d77069\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"
Jul 30 04:31:23.984162 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.984068    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd46d82e-52b7-4b2e-882d-edbc97582a1d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj\" (UID: \"bd46d82e-52b7-4b2e-882d-edbc97582a1d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"
Jul 30 04:31:23.984162 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.984103    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8np4\" (UniqueName: \"kubernetes.io/projected/bd46d82e-52b7-4b2e-882d-edbc97582a1d-kube-api-access-w8np4\") pod \"payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj\" (UID: \"bd46d82e-52b7-4b2e-882d-edbc97582a1d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"
Jul 30 04:31:23.984872 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.984854    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd46d82e-52b7-4b2e-882d-edbc97582a1d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj\" (UID: \"bd46d82e-52b7-4b2e-882d-edbc97582a1d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"
Jul 30 04:31:23.991628 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:23.991599    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8np4\" (UniqueName: \"kubernetes.io/projected/bd46d82e-52b7-4b2e-882d-edbc97582a1d-kube-api-access-w8np4\") pod \"payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj\" (UID: \"bd46d82e-52b7-4b2e-882d-edbc97582a1d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"
Jul 30 04:31:24.052206 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:24.052180    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:31:24.085751 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:24.085716    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecc7bb6a-6fa6-4237-84f7-030136d77069-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q\" (UID: \"ecc7bb6a-6fa6-4237-84f7-030136d77069\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"
Jul 30 04:31:24.085900 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:24.085779    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xqnt\" (UniqueName: \"kubernetes.io/projected/ecc7bb6a-6fa6-4237-84f7-030136d77069-kube-api-access-9xqnt\") pod \"payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q\" (UID: \"ecc7bb6a-6fa6-4237-84f7-030136d77069\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"
Jul 30 04:31:24.086447 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:24.086421    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecc7bb6a-6fa6-4237-84f7-030136d77069-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q\" (UID: \"ecc7bb6a-6fa6-4237-84f7-030136d77069\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"
Jul 30 04:31:24.093680 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:24.093655    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xqnt\" (UniqueName: \"kubernetes.io/projected/ecc7bb6a-6fa6-4237-84f7-030136d77069-kube-api-access-9xqnt\") pod \"payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q\" (UID: \"ecc7bb6a-6fa6-4237-84f7-030136d77069\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"
Jul 30 04:31:24.107302 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:24.107278    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"
Jul 30 04:31:24.135759 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:24.135728    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"
Jul 30 04:31:24.196720 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:31:24.196663    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc53b5a_d515_4c21_b7bd_fa28d8c08f84.slice/crio-7a16f0f46354041d0e7726d2cd0170c9015faf077bd5f8ef33bf6e0c1dd6273e WatchSource:0}: Error finding container 7a16f0f46354041d0e7726d2cd0170c9015faf077bd5f8ef33bf6e0c1dd6273e: Status 404 returned error can't find the container with id 7a16f0f46354041d0e7726d2cd0170c9015faf077bd5f8ef33bf6e0c1dd6273e
Jul 30 04:31:24.197328 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:24.196830    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"]
Jul 30 04:31:24.264986 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:24.264958    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"]
Jul 30 04:31:24.267272 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:31:24.267243    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd46d82e_52b7_4b2e_882d_edbc97582a1d.slice/crio-f0aec396e8f8a7a69ffcbdbba4b9533d1b063bb7c8463bf7568df283d8289467 WatchSource:0}: Error finding container f0aec396e8f8a7a69ffcbdbba4b9533d1b063bb7c8463bf7568df283d8289467: Status 404 returned error can't find the container with id f0aec396e8f8a7a69ffcbdbba4b9533d1b063bb7c8463bf7568df283d8289467
Jul 30 04:31:24.498011 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:24.497985    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"]
Jul 30 04:31:24.499419 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:31:24.499393    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc7bb6a_6fa6_4237_84f7_030136d77069.slice/crio-28a0200ac841bbbe16d11a6f98015ab0a6fa86265c5e3eed79ae84662e0fb98e WatchSource:0}: Error finding container 28a0200ac841bbbe16d11a6f98015ab0a6fa86265c5e3eed79ae84662e0fb98e: Status 404 returned error can't find the container with id 28a0200ac841bbbe16d11a6f98015ab0a6fa86265c5e3eed79ae84662e0fb98e
Jul 30 04:31:25.171292 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:25.171259    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7" event={"ID":"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84","Type":"ContainerStarted","Data":"a56773a753f1ae3fae382da711d95962bcd41619cd840a05a5205c02286695ab"}
Jul 30 04:31:25.171292 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:25.171310    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7" event={"ID":"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84","Type":"ContainerStarted","Data":"7a16f0f46354041d0e7726d2cd0170c9015faf077bd5f8ef33bf6e0c1dd6273e"}
Jul 30 04:31:25.171615 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:25.171354    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:31:25.172859 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:25.172837    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q" event={"ID":"ecc7bb6a-6fa6-4237-84f7-030136d77069","Type":"ContainerStarted","Data":"d87f70ffa6dbe3c945ea9e42b316e4719ae0d06cf617d15fe512bb2bbf203d56"}
Jul 30 04:31:25.172986 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:25.172876    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q" event={"ID":"ecc7bb6a-6fa6-4237-84f7-030136d77069","Type":"ContainerStarted","Data":"28a0200ac841bbbe16d11a6f98015ab0a6fa86265c5e3eed79ae84662e0fb98e"}
Jul 30 04:31:25.172986 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:25.172908    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"
Jul 30 04:31:25.174309 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:25.174288    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj" event={"ID":"bd46d82e-52b7-4b2e-882d-edbc97582a1d","Type":"ContainerStarted","Data":"0cce40f9eeb4bc53b2832f1865fa57c08fc23898c912ea0436f16cadf763b8f9"}
Jul 30 04:31:25.174402 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:25.174322    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj" event={"ID":"bd46d82e-52b7-4b2e-882d-edbc97582a1d","Type":"ContainerStarted","Data":"f0aec396e8f8a7a69ffcbdbba4b9533d1b063bb7c8463bf7568df283d8289467"}
Jul 30 04:31:25.174402 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:25.174352    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"
Jul 30 04:31:25.188812 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:25.188774    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7" podStartSLOduration=1.691360817 podStartE2EDuration="2.188761074s" totalImagesPullingTime="497.400257ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:31:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:31:25.18705405 +0000 UTC m=+2784.225765071" watchObservedRunningTime="2026-07-30 04:31:25.188761074 +0000 UTC m=+2784.227472130"
Jul 30 04:31:25.202915 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:25.202875    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q" podStartSLOduration=2.202858876 podStartE2EDuration="2.202858876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:31:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:31:25.200958974 +0000 UTC m=+2784.239669996" watchObservedRunningTime="2026-07-30 04:31:25.202858876 +0000 UTC m=+2784.241569898"
Jul 30 04:31:25.216145 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:25.216111    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj" podStartSLOduration=2.216102515 podStartE2EDuration="2.216102515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:31:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:31:25.215345397 +0000 UTC m=+2784.254056416" watchObservedRunningTime="2026-07-30 04:31:25.216102515 +0000 UTC m=+2784.254813534"
Jul 30 04:31:31.185458 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:31.185430    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:31:36.182047 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:36.182014    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"
Jul 30 04:31:36.182444 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:36.182101    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"
Jul 30 04:31:46.881431 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:46.881402    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:50.979130 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:31:50.979095    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:32:09.276670 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:32:09.276624    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:32:58.378315 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:32:58.378203    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:34:48.445925 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:34:48.445832    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:35:01.676068 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:01.676031    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:35:01.693966 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:01.693936    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:35:07.797407 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:07.797373    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:35:41.147569 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:41.147515    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-htsk6"]
Jul 30 04:35:41.150835 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:41.150816    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-htsk6"
Jul 30 04:35:41.152841 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:41.152818    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 04:35:41.153392 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:41.153375    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-swmvp\""
Jul 30 04:35:41.153479 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:41.153377    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 04:35:41.167368 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:41.167347    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-htsk6"]
Jul 30 04:35:41.202927 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:41.202903    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mhg2\" (UniqueName: \"kubernetes.io/projected/03284c3a-b7ee-443d-a412-8895ddfb7c73-kube-api-access-4mhg2\") pod \"kuadrant-operator-catalog-htsk6\" (UID: \"03284c3a-b7ee-443d-a412-8895ddfb7c73\") " pod="kuadrant-system/kuadrant-operator-catalog-htsk6"
Jul 30 04:35:41.303838 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:41.303814    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mhg2\" (UniqueName: \"kubernetes.io/projected/03284c3a-b7ee-443d-a412-8895ddfb7c73-kube-api-access-4mhg2\") pod \"kuadrant-operator-catalog-htsk6\" (UID: \"03284c3a-b7ee-443d-a412-8895ddfb7c73\") " pod="kuadrant-system/kuadrant-operator-catalog-htsk6"
Jul 30 04:35:41.312063 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:41.312042    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mhg2\" (UniqueName: \"kubernetes.io/projected/03284c3a-b7ee-443d-a412-8895ddfb7c73-kube-api-access-4mhg2\") pod \"kuadrant-operator-catalog-htsk6\" (UID: \"03284c3a-b7ee-443d-a412-8895ddfb7c73\") " pod="kuadrant-system/kuadrant-operator-catalog-htsk6"
Jul 30 04:35:41.460030 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:41.459954    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-htsk6"
Jul 30 04:35:41.595774 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:41.595745    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-htsk6"]
Jul 30 04:35:41.597150 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:35:41.597119    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03284c3a_b7ee_443d_a412_8895ddfb7c73.slice/crio-f232a63f78a1dded64af4fca4f2aefddd240e89b6c749e3fab5622d7b39bf766 WatchSource:0}: Error finding container f232a63f78a1dded64af4fca4f2aefddd240e89b6c749e3fab5622d7b39bf766: Status 404 returned error can't find the container with id f232a63f78a1dded64af4fca4f2aefddd240e89b6c749e3fab5622d7b39bf766
Jul 30 04:35:41.598565 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:41.598546    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 04:35:42.483767 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:42.483725    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-htsk6" event={"ID":"03284c3a-b7ee-443d-a412-8895ddfb7c73","Type":"ContainerStarted","Data":"f232a63f78a1dded64af4fca4f2aefddd240e89b6c749e3fab5622d7b39bf766"}
Jul 30 04:35:43.489984 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:43.489948    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-htsk6" event={"ID":"03284c3a-b7ee-443d-a412-8895ddfb7c73","Type":"ContainerStarted","Data":"a228f32ba141ed90c1283b91402de4edf43bc3dc9c119d9f82cc5da237d36253"}
Jul 30 04:35:43.504718 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:43.504670    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-htsk6" podStartSLOduration=1.158358798 podStartE2EDuration="2.504655795s" totalImagesPullingTime="1.346296997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:35:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:35:43.502921115 +0000 UTC m=+3042.541632134" watchObservedRunningTime="2026-07-30 04:35:43.504655795 +0000 UTC m=+3042.543366814"
Jul 30 04:35:51.466172 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:51.466094    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-htsk6"
Jul 30 04:35:51.466172 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:51.466143    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-htsk6"
Jul 30 04:35:51.484443 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:51.484418    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-htsk6"
Jul 30 04:35:51.548171 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:51.548148    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-htsk6"
Jul 30 04:35:51.709055 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:51.709025    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-htsk6"]
Jul 30 04:35:53.536302 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:53.536264    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-htsk6" podUID="03284c3a-b7ee-443d-a412-8895ddfb7c73" containerName="registry-server" containerID="cri-o://a228f32ba141ed90c1283b91402de4edf43bc3dc9c119d9f82cc5da237d36253" gracePeriod=2
Jul 30 04:35:53.787384 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:53.787321    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-htsk6"
Jul 30 04:35:53.904150 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:53.904119    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mhg2\" (UniqueName: \"kubernetes.io/projected/03284c3a-b7ee-443d-a412-8895ddfb7c73-kube-api-access-4mhg2\") pod \"03284c3a-b7ee-443d-a412-8895ddfb7c73\" (UID: \"03284c3a-b7ee-443d-a412-8895ddfb7c73\") "
Jul 30 04:35:53.906220 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:53.906197    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03284c3a-b7ee-443d-a412-8895ddfb7c73-kube-api-access-4mhg2" (OuterVolumeSpecName: "kube-api-access-4mhg2") pod "03284c3a-b7ee-443d-a412-8895ddfb7c73" (UID: "03284c3a-b7ee-443d-a412-8895ddfb7c73"). InnerVolumeSpecName "kube-api-access-4mhg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:35:54.005622 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:54.005593    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mhg2\" (UniqueName: \"kubernetes.io/projected/03284c3a-b7ee-443d-a412-8895ddfb7c73-kube-api-access-4mhg2\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:35:54.542485 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:54.542444    2569 generic.go:356] "Generic (PLEG): container finished" podID="03284c3a-b7ee-443d-a412-8895ddfb7c73" containerID="a228f32ba141ed90c1283b91402de4edf43bc3dc9c119d9f82cc5da237d36253" exitCode=0
Jul 30 04:35:54.542485 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:54.542487    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-htsk6"
Jul 30 04:35:54.543044 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:54.542500    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-htsk6" event={"ID":"03284c3a-b7ee-443d-a412-8895ddfb7c73","Type":"ContainerDied","Data":"a228f32ba141ed90c1283b91402de4edf43bc3dc9c119d9f82cc5da237d36253"}
Jul 30 04:35:54.543044 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:54.542578    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-htsk6" event={"ID":"03284c3a-b7ee-443d-a412-8895ddfb7c73","Type":"ContainerDied","Data":"f232a63f78a1dded64af4fca4f2aefddd240e89b6c749e3fab5622d7b39bf766"}
Jul 30 04:35:54.543044 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:54.542601    2569 scope.go:117] "RemoveContainer" containerID="a228f32ba141ed90c1283b91402de4edf43bc3dc9c119d9f82cc5da237d36253"
Jul 30 04:35:54.552576 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:54.552557    2569 scope.go:117] "RemoveContainer" containerID="a228f32ba141ed90c1283b91402de4edf43bc3dc9c119d9f82cc5da237d36253"
Jul 30 04:35:54.552825 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:35:54.552807    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a228f32ba141ed90c1283b91402de4edf43bc3dc9c119d9f82cc5da237d36253\": container with ID starting with a228f32ba141ed90c1283b91402de4edf43bc3dc9c119d9f82cc5da237d36253 not found: ID does not exist" containerID="a228f32ba141ed90c1283b91402de4edf43bc3dc9c119d9f82cc5da237d36253"
Jul 30 04:35:54.552880 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:54.552833    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a228f32ba141ed90c1283b91402de4edf43bc3dc9c119d9f82cc5da237d36253"} err="failed to get container status \"a228f32ba141ed90c1283b91402de4edf43bc3dc9c119d9f82cc5da237d36253\": rpc error: code = NotFound desc = could not find container \"a228f32ba141ed90c1283b91402de4edf43bc3dc9c119d9f82cc5da237d36253\": container with ID starting with a228f32ba141ed90c1283b91402de4edf43bc3dc9c119d9f82cc5da237d36253 not found: ID does not exist"
Jul 30 04:35:54.562809 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:54.562786    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-htsk6"]
Jul 30 04:35:54.564842 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:54.564823    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-htsk6"]
Jul 30 04:35:55.466141 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:35:55.466105    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03284c3a-b7ee-443d-a412-8895ddfb7c73" path="/var/lib/kubelet/pods/03284c3a-b7ee-443d-a412-8895ddfb7c73/volumes"
Jul 30 04:37:24.940434 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:37:24.940328    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:37:25.855324 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:37:25.855283    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:37:40.283955 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:37:40.283916    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:38:33.764610 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:38:33.764574    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:01.716342 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:40:01.716308    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:40:01.729067 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:40:01.729046    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:40:12.879820 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:40:12.879775    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:13.973346 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:40:13.973306    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:22.871544 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:40:22.871483    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:38.986246 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:40:38.986215    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:40.615899 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:40:40.615863    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:58.285948 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:40:58.285900    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:42:50.665272 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:50.665226    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:42:51.556249 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:51.556198    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:42:55.561859 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:55.561821    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7753-140636230607664"]
Jul 30 04:42:55.562340 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:55.562318    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03284c3a-b7ee-443d-a412-8895ddfb7c73" containerName="registry-server"
Jul 30 04:42:55.562418 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:55.562345    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="03284c3a-b7ee-443d-a412-8895ddfb7c73" containerName="registry-server"
Jul 30 04:42:55.562477 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:55.562458    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="03284c3a-b7ee-443d-a412-8895ddfb7c73" containerName="registry-server"
Jul 30 04:42:55.565371 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:55.565350    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-140636230607664"
Jul 30 04:42:55.571867 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:55.571668    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dzc85\""
Jul 30 04:42:55.591258 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:55.591235    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7753-140636230607664"]
Jul 30 04:42:55.662592 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:55.662563    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkztw\" (UniqueName: \"kubernetes.io/projected/98734743-6f73-48f5-8385-dfe7ba2504c0-kube-api-access-tkztw\") pod \"test-curl-7753-140636230607664\" (UID: \"98734743-6f73-48f5-8385-dfe7ba2504c0\") " pod="opendatahub/test-curl-7753-140636230607664"
Jul 30 04:42:55.763402 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:55.763374    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkztw\" (UniqueName: \"kubernetes.io/projected/98734743-6f73-48f5-8385-dfe7ba2504c0-kube-api-access-tkztw\") pod \"test-curl-7753-140636230607664\" (UID: \"98734743-6f73-48f5-8385-dfe7ba2504c0\") " pod="opendatahub/test-curl-7753-140636230607664"
Jul 30 04:42:55.771317 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:55.771291    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkztw\" (UniqueName: \"kubernetes.io/projected/98734743-6f73-48f5-8385-dfe7ba2504c0-kube-api-access-tkztw\") pod \"test-curl-7753-140636230607664\" (UID: \"98734743-6f73-48f5-8385-dfe7ba2504c0\") " pod="opendatahub/test-curl-7753-140636230607664"
Jul 30 04:42:55.874625 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:55.874565    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-140636230607664"
Jul 30 04:42:56.212959 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:56.212937    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7753-140636230607664"]
Jul 30 04:42:56.215089 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:42:56.215062    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98734743_6f73_48f5_8385_dfe7ba2504c0.slice/crio-cf86ba98eccd0a86e1ca241d857c24264184daadcbc854f7e7920547df89af23 WatchSource:0}: Error finding container cf86ba98eccd0a86e1ca241d857c24264184daadcbc854f7e7920547df89af23: Status 404 returned error can't find the container with id cf86ba98eccd0a86e1ca241d857c24264184daadcbc854f7e7920547df89af23
Jul 30 04:42:56.216242 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:56.216221    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 04:42:56.679918 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:56.679885    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7753-140636230607664" event={"ID":"98734743-6f73-48f5-8385-dfe7ba2504c0","Type":"ContainerStarted","Data":"cf86ba98eccd0a86e1ca241d857c24264184daadcbc854f7e7920547df89af23"}
Jul 30 04:42:57.686111 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:57.686079    2569 generic.go:356] "Generic (PLEG): container finished" podID="98734743-6f73-48f5-8385-dfe7ba2504c0" containerID="158abe4a9a2458857d64efe3670be359ae24d3df39cd54adc1fc2cd7bd0f564e" exitCode=0
Jul 30 04:42:57.686501 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:57.686166    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7753-140636230607664" event={"ID":"98734743-6f73-48f5-8385-dfe7ba2504c0","Type":"ContainerDied","Data":"158abe4a9a2458857d64efe3670be359ae24d3df39cd54adc1fc2cd7bd0f564e"}
Jul 30 04:42:58.826354 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:58.826325    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-140636230607664"
Jul 30 04:42:58.846521 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:58.846492    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7753-140636230607664_98734743-6f73-48f5-8385-dfe7ba2504c0/test-curl-7753-140636230607664/0.log"
Jul 30 04:42:58.872259 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:58.872233    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7753-140636230607664"]
Jul 30 04:42:58.874242 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:58.874221    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7753-140636230607664"]
Jul 30 04:42:58.888855 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:58.888832    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkztw\" (UniqueName: \"kubernetes.io/projected/98734743-6f73-48f5-8385-dfe7ba2504c0-kube-api-access-tkztw\") pod \"98734743-6f73-48f5-8385-dfe7ba2504c0\" (UID: \"98734743-6f73-48f5-8385-dfe7ba2504c0\") "
Jul 30 04:42:58.890980 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:58.890954    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98734743-6f73-48f5-8385-dfe7ba2504c0-kube-api-access-tkztw" (OuterVolumeSpecName: "kube-api-access-tkztw") pod "98734743-6f73-48f5-8385-dfe7ba2504c0" (UID: "98734743-6f73-48f5-8385-dfe7ba2504c0"). InnerVolumeSpecName "kube-api-access-tkztw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:42:58.989863 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:58.989794    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkztw\" (UniqueName: \"kubernetes.io/projected/98734743-6f73-48f5-8385-dfe7ba2504c0-kube-api-access-tkztw\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:42:59.037304 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.037276    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7753-140636230491280"]
Jul 30 04:42:59.037650 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.037638    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98734743-6f73-48f5-8385-dfe7ba2504c0" containerName="test-curl-7753-140636230607664"
Jul 30 04:42:59.037705 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.037652    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="98734743-6f73-48f5-8385-dfe7ba2504c0" containerName="test-curl-7753-140636230607664"
Jul 30 04:42:59.037756 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.037725    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="98734743-6f73-48f5-8385-dfe7ba2504c0" containerName="test-curl-7753-140636230607664"
Jul 30 04:42:59.040665 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.040651    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-140636230491280"
Jul 30 04:42:59.046360 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.046340    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7753-140636230491280"]
Jul 30 04:42:59.191054 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.191021    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4g2g\" (UniqueName: \"kubernetes.io/projected/6bd22a16-eacb-405a-a6df-691576f233da-kube-api-access-j4g2g\") pod \"test-curl-7753-140636230491280\" (UID: \"6bd22a16-eacb-405a-a6df-691576f233da\") " pod="opendatahub/test-curl-7753-140636230491280"
Jul 30 04:42:59.291461 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.291431    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4g2g\" (UniqueName: \"kubernetes.io/projected/6bd22a16-eacb-405a-a6df-691576f233da-kube-api-access-j4g2g\") pod \"test-curl-7753-140636230491280\" (UID: \"6bd22a16-eacb-405a-a6df-691576f233da\") " pod="opendatahub/test-curl-7753-140636230491280"
Jul 30 04:42:59.299298 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.299272    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4g2g\" (UniqueName: \"kubernetes.io/projected/6bd22a16-eacb-405a-a6df-691576f233da-kube-api-access-j4g2g\") pod \"test-curl-7753-140636230491280\" (UID: \"6bd22a16-eacb-405a-a6df-691576f233da\") " pod="opendatahub/test-curl-7753-140636230491280"
Jul 30 04:42:59.350428 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.350398    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-140636230491280"
Jul 30 04:42:59.467292 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.467260    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98734743-6f73-48f5-8385-dfe7ba2504c0" path="/var/lib/kubelet/pods/98734743-6f73-48f5-8385-dfe7ba2504c0/volumes"
Jul 30 04:42:59.481710 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.481688    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7753-140636230491280"]
Jul 30 04:42:59.482318 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:42:59.482300    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bd22a16_eacb_405a_a6df_691576f233da.slice/crio-5659863a7f01d13e21c634f944b3f3649dc8336431050e75a5c6e3a15dc24c19 WatchSource:0}: Error finding container 5659863a7f01d13e21c634f944b3f3649dc8336431050e75a5c6e3a15dc24c19: Status 404 returned error can't find the container with id 5659863a7f01d13e21c634f944b3f3649dc8336431050e75a5c6e3a15dc24c19
Jul 30 04:42:59.698861 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.698770    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-140636230607664"
Jul 30 04:42:59.698861 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.698777    2569 scope.go:117] "RemoveContainer" containerID="158abe4a9a2458857d64efe3670be359ae24d3df39cd54adc1fc2cd7bd0f564e"
Jul 30 04:42:59.700124 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:42:59.699975    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7753-140636230491280" event={"ID":"6bd22a16-eacb-405a-a6df-691576f233da","Type":"ContainerStarted","Data":"5659863a7f01d13e21c634f944b3f3649dc8336431050e75a5c6e3a15dc24c19"}
Jul 30 04:43:00.706710 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:00.706679    2569 generic.go:356] "Generic (PLEG): container finished" podID="6bd22a16-eacb-405a-a6df-691576f233da" containerID="a9e6d30f5267497579cc38bfcdbf21aef2ecf3edd5271a65361cce2352b8da20" exitCode=0
Jul 30 04:43:00.707083 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:00.706734    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7753-140636230491280" event={"ID":"6bd22a16-eacb-405a-a6df-691576f233da","Type":"ContainerDied","Data":"a9e6d30f5267497579cc38bfcdbf21aef2ecf3edd5271a65361cce2352b8da20"}
Jul 30 04:43:01.846846 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:01.846819    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-140636230491280"
Jul 30 04:43:01.865077 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:01.865055    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7753-140636230491280_6bd22a16-eacb-405a-a6df-691576f233da/test-curl-7753-140636230491280/0.log"
Jul 30 04:43:01.889371 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:01.889344    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7753-140636230491280"]
Jul 30 04:43:01.891359 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:01.891340    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7753-140636230491280"]
Jul 30 04:43:01.913041 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:01.913018    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4g2g\" (UniqueName: \"kubernetes.io/projected/6bd22a16-eacb-405a-a6df-691576f233da-kube-api-access-j4g2g\") pod \"6bd22a16-eacb-405a-a6df-691576f233da\" (UID: \"6bd22a16-eacb-405a-a6df-691576f233da\") "
Jul 30 04:43:01.915097 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:01.915073    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bd22a16-eacb-405a-a6df-691576f233da-kube-api-access-j4g2g" (OuterVolumeSpecName: "kube-api-access-j4g2g") pod "6bd22a16-eacb-405a-a6df-691576f233da" (UID: "6bd22a16-eacb-405a-a6df-691576f233da"). InnerVolumeSpecName "kube-api-access-j4g2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:43:02.014323 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:02.014241    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4g2g\" (UniqueName: \"kubernetes.io/projected/6bd22a16-eacb-405a-a6df-691576f233da-kube-api-access-j4g2g\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:43:02.717556 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:02.717506    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-140636230491280"
Jul 30 04:43:02.717556 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:02.717524    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5659863a7f01d13e21c634f944b3f3649dc8336431050e75a5c6e3a15dc24c19"
Jul 30 04:43:02.726853 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:43:02.726826    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7753-140636230491280\" is forbidden: User \"system:node:ip-10-0-136-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-104.ec2.internal' and this object" podUID="6bd22a16-eacb-405a-a6df-691576f233da" pod="opendatahub/test-curl-7753-140636230491280"
Jul 30 04:43:03.466140 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:03.466109    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bd22a16-eacb-405a-a6df-691576f233da" path="/var/lib/kubelet/pods/6bd22a16-eacb-405a-a6df-691576f233da/volumes"
Jul 30 04:43:04.577223 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:04.577189    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7753-140636230459440"]
Jul 30 04:43:04.577620 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:04.577519    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bd22a16-eacb-405a-a6df-691576f233da" containerName="test-curl-7753-140636230491280"
Jul 30 04:43:04.577620 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:04.577545    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bd22a16-eacb-405a-a6df-691576f233da" containerName="test-curl-7753-140636230491280"
Jul 30 04:43:04.577698 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:04.577642    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bd22a16-eacb-405a-a6df-691576f233da" containerName="test-curl-7753-140636230491280"
Jul 30 04:43:04.581878 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:04.581862    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-140636230459440"
Jul 30 04:43:04.584322 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:04.584297    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dzc85\""
Jul 30 04:43:04.586280 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:04.586258    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7753-140636230459440"]
Jul 30 04:43:04.736267 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:04.736234    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhw69\" (UniqueName: \"kubernetes.io/projected/b9830fe7-0881-4428-b59f-bd564f5a9bc9-kube-api-access-hhw69\") pod \"test-curl-7753-140636230459440\" (UID: \"b9830fe7-0881-4428-b59f-bd564f5a9bc9\") " pod="opendatahub/test-curl-7753-140636230459440"
Jul 30 04:43:04.837341 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:04.837267    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhw69\" (UniqueName: \"kubernetes.io/projected/b9830fe7-0881-4428-b59f-bd564f5a9bc9-kube-api-access-hhw69\") pod \"test-curl-7753-140636230459440\" (UID: \"b9830fe7-0881-4428-b59f-bd564f5a9bc9\") " pod="opendatahub/test-curl-7753-140636230459440"
Jul 30 04:43:04.845073 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:04.845042    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhw69\" (UniqueName: \"kubernetes.io/projected/b9830fe7-0881-4428-b59f-bd564f5a9bc9-kube-api-access-hhw69\") pod \"test-curl-7753-140636230459440\" (UID: \"b9830fe7-0881-4428-b59f-bd564f5a9bc9\") " pod="opendatahub/test-curl-7753-140636230459440"
Jul 30 04:43:04.893465 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:04.893441    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-140636230459440"
Jul 30 04:43:05.024117 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:05.024092    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7753-140636230459440"]
Jul 30 04:43:05.025613 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:43:05.025585    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9830fe7_0881_4428_b59f_bd564f5a9bc9.slice/crio-8313ef894cd7e7540a7957214bc9dda2ebe99b951b7ea8cf1df666888533b3e9 WatchSource:0}: Error finding container 8313ef894cd7e7540a7957214bc9dda2ebe99b951b7ea8cf1df666888533b3e9: Status 404 returned error can't find the container with id 8313ef894cd7e7540a7957214bc9dda2ebe99b951b7ea8cf1df666888533b3e9
Jul 30 04:43:05.735180 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:05.735095    2569 generic.go:356] "Generic (PLEG): container finished" podID="b9830fe7-0881-4428-b59f-bd564f5a9bc9" containerID="3329fa3794fd6e65d999fc7ce8f2e5eba1fe59f069c4587d3391ed3c49b5c73d" exitCode=0
Jul 30 04:43:05.735180 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:05.735135    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7753-140636230459440" event={"ID":"b9830fe7-0881-4428-b59f-bd564f5a9bc9","Type":"ContainerDied","Data":"3329fa3794fd6e65d999fc7ce8f2e5eba1fe59f069c4587d3391ed3c49b5c73d"}
Jul 30 04:43:05.735180 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:05.735162    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7753-140636230459440" event={"ID":"b9830fe7-0881-4428-b59f-bd564f5a9bc9","Type":"ContainerStarted","Data":"8313ef894cd7e7540a7957214bc9dda2ebe99b951b7ea8cf1df666888533b3e9"}
Jul 30 04:43:06.872102 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:06.872081    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-140636230459440"
Jul 30 04:43:06.890254 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:06.890227    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7753-140636230459440_b9830fe7-0881-4428-b59f-bd564f5a9bc9/test-curl-7753-140636230459440/0.log"
Jul 30 04:43:06.915395 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:06.915355    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7753-140636230459440"]
Jul 30 04:43:06.918028 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:06.918008    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7753-140636230459440"]
Jul 30 04:43:06.958178 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:06.958154    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhw69\" (UniqueName: \"kubernetes.io/projected/b9830fe7-0881-4428-b59f-bd564f5a9bc9-kube-api-access-hhw69\") pod \"b9830fe7-0881-4428-b59f-bd564f5a9bc9\" (UID: \"b9830fe7-0881-4428-b59f-bd564f5a9bc9\") "
Jul 30 04:43:06.960243 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:06.960223    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9830fe7-0881-4428-b59f-bd564f5a9bc9-kube-api-access-hhw69" (OuterVolumeSpecName: "kube-api-access-hhw69") pod "b9830fe7-0881-4428-b59f-bd564f5a9bc9" (UID: "b9830fe7-0881-4428-b59f-bd564f5a9bc9"). InnerVolumeSpecName "kube-api-access-hhw69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:43:07.059075 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:07.059041    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhw69\" (UniqueName: \"kubernetes.io/projected/b9830fe7-0881-4428-b59f-bd564f5a9bc9-kube-api-access-hhw69\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:43:07.465344 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:07.465250    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9830fe7-0881-4428-b59f-bd564f5a9bc9" path="/var/lib/kubelet/pods/b9830fe7-0881-4428-b59f-bd564f5a9bc9/volumes"
Jul 30 04:43:07.747170 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:07.747085    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-140636230459440"
Jul 30 04:43:07.747170 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:07.747133    2569 scope.go:117] "RemoveContainer" containerID="3329fa3794fd6e65d999fc7ce8f2e5eba1fe59f069c4587d3391ed3c49b5c73d"
Jul 30 04:43:09.963845 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:09.963809    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:10.272791 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:10.272759    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:10.480191 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:10.480159    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:13.667878 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:13.667839    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:13.775121 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:13.775090    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:14.806488 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:14.806458    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:15.006847 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.006805    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:15.306346 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.306314    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:15.410623 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.410580    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:15.412019 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.411998    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"]
Jul 30 04:43:15.412367 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.412355    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9830fe7-0881-4428-b59f-bd564f5a9bc9" containerName="test-curl-7753-140636230459440"
Jul 30 04:43:15.412408 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.412369    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9830fe7-0881-4428-b59f-bd564f5a9bc9" containerName="test-curl-7753-140636230459440"
Jul 30 04:43:15.412443 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.412438    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9830fe7-0881-4428-b59f-bd564f5a9bc9" containerName="test-curl-7753-140636230459440"
Jul 30 04:43:15.415436 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.415422    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"
Jul 30 04:43:15.417576 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.417556    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-a1808a\""
Jul 30 04:43:15.417708 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.417599    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-a1808a-dockercfg-m4khh\""
Jul 30 04:43:15.425512 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.425482    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"]
Jul 30 04:43:15.437243 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.437216    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"]
Jul 30 04:43:15.440605 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.440590    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"
Jul 30 04:43:15.448758 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.448737    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"]
Jul 30 04:43:15.536344 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.536313    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/966e3746-6558-4c91-9329-595ce491b633-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9\" (UID: \"966e3746-6558-4c91-9329-595ce491b633\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"
Jul 30 04:43:15.536519 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.536398    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe21cc92-8356-4828-834c-854b306ef3c0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk\" (UID: \"fe21cc92-8356-4828-834c-854b306ef3c0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"
Jul 30 04:43:15.536519 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.536426    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz4g5\" (UniqueName: \"kubernetes.io/projected/fe21cc92-8356-4828-834c-854b306ef3c0-kube-api-access-jz4g5\") pod \"payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk\" (UID: \"fe21cc92-8356-4828-834c-854b306ef3c0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"
Jul 30 04:43:15.536519 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.536492    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7l5t\" (UniqueName: \"kubernetes.io/projected/966e3746-6558-4c91-9329-595ce491b633-kube-api-access-c7l5t\") pod \"payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9\" (UID: \"966e3746-6558-4c91-9329-595ce491b633\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"
Jul 30 04:43:15.636994 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.636921    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/966e3746-6558-4c91-9329-595ce491b633-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9\" (UID: \"966e3746-6558-4c91-9329-595ce491b633\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"
Jul 30 04:43:15.636994 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.636975    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe21cc92-8356-4828-834c-854b306ef3c0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk\" (UID: \"fe21cc92-8356-4828-834c-854b306ef3c0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"
Jul 30 04:43:15.636994 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.636994    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4g5\" (UniqueName: \"kubernetes.io/projected/fe21cc92-8356-4828-834c-854b306ef3c0-kube-api-access-jz4g5\") pod \"payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk\" (UID: \"fe21cc92-8356-4828-834c-854b306ef3c0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"
Jul 30 04:43:15.637257 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.637171    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7l5t\" (UniqueName: \"kubernetes.io/projected/966e3746-6558-4c91-9329-595ce491b633-kube-api-access-c7l5t\") pod \"payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9\" (UID: \"966e3746-6558-4c91-9329-595ce491b633\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"
Jul 30 04:43:15.637727 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.637707    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/966e3746-6558-4c91-9329-595ce491b633-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9\" (UID: \"966e3746-6558-4c91-9329-595ce491b633\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"
Jul 30 04:43:15.637819 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.637706    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe21cc92-8356-4828-834c-854b306ef3c0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk\" (UID: \"fe21cc92-8356-4828-834c-854b306ef3c0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"
Jul 30 04:43:15.649022 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.648994    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz4g5\" (UniqueName: \"kubernetes.io/projected/fe21cc92-8356-4828-834c-854b306ef3c0-kube-api-access-jz4g5\") pod \"payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk\" (UID: \"fe21cc92-8356-4828-834c-854b306ef3c0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"
Jul 30 04:43:15.649120 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.649035    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7l5t\" (UniqueName: \"kubernetes.io/projected/966e3746-6558-4c91-9329-595ce491b633-kube-api-access-c7l5t\") pod \"payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9\" (UID: \"966e3746-6558-4c91-9329-595ce491b633\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"
Jul 30 04:43:15.726524 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.726500    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"
Jul 30 04:43:15.751964 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.751941    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"
Jul 30 04:43:15.876310 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.876225    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"]
Jul 30 04:43:15.878553 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:43:15.878504    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod966e3746_6558_4c91_9329_595ce491b633.slice/crio-f6dd98b899cfa5c988cd45381580b230bc67604b3a920d8d5f76439a2c693c38 WatchSource:0}: Error finding container f6dd98b899cfa5c988cd45381580b230bc67604b3a920d8d5f76439a2c693c38: Status 404 returned error can't find the container with id f6dd98b899cfa5c988cd45381580b230bc67604b3a920d8d5f76439a2c693c38
Jul 30 04:43:15.905620 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:15.905597    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"]
Jul 30 04:43:15.915561 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:43:15.915516    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe21cc92_8356_4828_834c_854b306ef3c0.slice/crio-30f911ee3a63850a03f0c2f8eebbeaa12a354b8b296c41bc264035cd71730bc7 WatchSource:0}: Error finding container 30f911ee3a63850a03f0c2f8eebbeaa12a354b8b296c41bc264035cd71730bc7: Status 404 returned error can't find the container with id 30f911ee3a63850a03f0c2f8eebbeaa12a354b8b296c41bc264035cd71730bc7
Jul 30 04:43:16.800602 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:16.800568    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk" event={"ID":"fe21cc92-8356-4828-834c-854b306ef3c0","Type":"ContainerStarted","Data":"c8659e01784d536865c56f31fff3c691c9a95d7bf4ddf8d62a097dd6e5dec057"}
Jul 30 04:43:16.800808 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:16.800630    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"
Jul 30 04:43:16.800808 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:16.800656    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk" event={"ID":"fe21cc92-8356-4828-834c-854b306ef3c0","Type":"ContainerStarted","Data":"30f911ee3a63850a03f0c2f8eebbeaa12a354b8b296c41bc264035cd71730bc7"}
Jul 30 04:43:16.802105 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:16.802077    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9" event={"ID":"966e3746-6558-4c91-9329-595ce491b633","Type":"ContainerStarted","Data":"fb8b7aac45cf797be6e3f92b5c398ac679f98e8b1c7256c694bd493929b266b3"}
Jul 30 04:43:16.802219 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:16.802120    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9" event={"ID":"966e3746-6558-4c91-9329-595ce491b633","Type":"ContainerStarted","Data":"f6dd98b899cfa5c988cd45381580b230bc67604b3a920d8d5f76439a2c693c38"}
Jul 30 04:43:16.802219 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:16.802179    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"
Jul 30 04:43:16.816829 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:16.816791    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk" podStartSLOduration=1.816778676 podStartE2EDuration="1.816778676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:43:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:43:16.814339537 +0000 UTC m=+3495.853050555" watchObservedRunningTime="2026-07-30 04:43:16.816778676 +0000 UTC m=+3495.855489694"
Jul 30 04:43:16.829867 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:16.829829    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9" podStartSLOduration=1.829815376 podStartE2EDuration="1.829815376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:43:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:43:16.828095994 +0000 UTC m=+3495.866807013" watchObservedRunningTime="2026-07-30 04:43:16.829815376 +0000 UTC m=+3495.868526395"
Jul 30 04:43:27.808719 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:27.808641    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"
Jul 30 04:43:27.809169 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:27.808835    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"
Jul 30 04:43:29.259776 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:29.259740    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:29.361185 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:29.361144    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:38.673572 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:38.673509    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:39.362636 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:39.362604    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:39.753945 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:39.753864    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:40.344371 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:40.344340    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:44.786986 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:44.786939    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:45.416690 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:45.416659    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:45.522744 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:45.522711    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:46.554096 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:46.554069    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:46.806994 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:46.806917    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:47.050939 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.050902    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:47.147600 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.147496    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:43:47.164330 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.164300    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"]
Jul 30 04:43:47.167735 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.167719    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"
Jul 30 04:43:47.169950 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.169930    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-eda6fc-dockercfg-xcbth\""
Jul 30 04:43:47.170061 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.169981    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-eda6fc\""
Jul 30 04:43:47.178365 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.178343    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"]
Jul 30 04:43:47.218724 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.218689    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"]
Jul 30 04:43:47.223380 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.223348    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"
Jul 30 04:43:47.229718 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.229694    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"]
Jul 30 04:43:47.295824 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.295797    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c12a586-90ad-49eb-a7f9-d040763f90ad-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4\" (UID: \"3c12a586-90ad-49eb-a7f9-d040763f90ad\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"
Jul 30 04:43:47.295979 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.295847    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxhlf\" (UniqueName: \"kubernetes.io/projected/3c12a586-90ad-49eb-a7f9-d040763f90ad-kube-api-access-qxhlf\") pod \"payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4\" (UID: \"3c12a586-90ad-49eb-a7f9-d040763f90ad\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"
Jul 30 04:43:47.295979 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.295927    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz7d4\" (UniqueName: \"kubernetes.io/projected/1275fbc0-ef79-4872-8963-65ba5f4587af-kube-api-access-vz7d4\") pod \"payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw\" (UID: \"1275fbc0-ef79-4872-8963-65ba5f4587af\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"
Jul 30 04:43:47.295979 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.295974    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1275fbc0-ef79-4872-8963-65ba5f4587af-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw\" (UID: \"1275fbc0-ef79-4872-8963-65ba5f4587af\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"
Jul 30 04:43:47.396919 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.396883    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxhlf\" (UniqueName: \"kubernetes.io/projected/3c12a586-90ad-49eb-a7f9-d040763f90ad-kube-api-access-qxhlf\") pod \"payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4\" (UID: \"3c12a586-90ad-49eb-a7f9-d040763f90ad\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"
Jul 30 04:43:47.397080 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.396945    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz7d4\" (UniqueName: \"kubernetes.io/projected/1275fbc0-ef79-4872-8963-65ba5f4587af-kube-api-access-vz7d4\") pod \"payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw\" (UID: \"1275fbc0-ef79-4872-8963-65ba5f4587af\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"
Jul 30 04:43:47.397080 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.396995    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1275fbc0-ef79-4872-8963-65ba5f4587af-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw\" (UID: \"1275fbc0-ef79-4872-8963-65ba5f4587af\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"
Jul 30 04:43:47.397080 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.397068    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c12a586-90ad-49eb-a7f9-d040763f90ad-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4\" (UID: \"3c12a586-90ad-49eb-a7f9-d040763f90ad\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"
Jul 30 04:43:47.397812 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.397746    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1275fbc0-ef79-4872-8963-65ba5f4587af-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw\" (UID: \"1275fbc0-ef79-4872-8963-65ba5f4587af\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"
Jul 30 04:43:47.397909 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.397834    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c12a586-90ad-49eb-a7f9-d040763f90ad-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4\" (UID: \"3c12a586-90ad-49eb-a7f9-d040763f90ad\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"
Jul 30 04:43:47.404201 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.404181    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz7d4\" (UniqueName: \"kubernetes.io/projected/1275fbc0-ef79-4872-8963-65ba5f4587af-kube-api-access-vz7d4\") pod \"payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw\" (UID: \"1275fbc0-ef79-4872-8963-65ba5f4587af\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"
Jul 30 04:43:47.404321 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.404189    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxhlf\" (UniqueName: \"kubernetes.io/projected/3c12a586-90ad-49eb-a7f9-d040763f90ad-kube-api-access-qxhlf\") pod \"payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4\" (UID: \"3c12a586-90ad-49eb-a7f9-d040763f90ad\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"
Jul 30 04:43:47.479468 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.479446    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"
Jul 30 04:43:47.536851 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.536822    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"
Jul 30 04:43:47.624981 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.624948    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"]
Jul 30 04:43:47.631064 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:43:47.631020    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c12a586_90ad_49eb_a7f9_d040763f90ad.slice/crio-60caf2ad0fcb07ce68000c04f852d4466c0def61057020724a077bb74c0442d9 WatchSource:0}: Error finding container 60caf2ad0fcb07ce68000c04f852d4466c0def61057020724a077bb74c0442d9: Status 404 returned error can't find the container with id 60caf2ad0fcb07ce68000c04f852d4466c0def61057020724a077bb74c0442d9
Jul 30 04:43:47.680828 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.680798    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"]
Jul 30 04:43:47.682292 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:43:47.682266    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1275fbc0_ef79_4872_8963_65ba5f4587af.slice/crio-438a5903e7105b6931fc32759ffd967aba6afd710312cd18819faace9820022d WatchSource:0}: Error finding container 438a5903e7105b6931fc32759ffd967aba6afd710312cd18819faace9820022d: Status 404 returned error can't find the container with id 438a5903e7105b6931fc32759ffd967aba6afd710312cd18819faace9820022d
Jul 30 04:43:47.972008 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.971907    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw" event={"ID":"1275fbc0-ef79-4872-8963-65ba5f4587af","Type":"ContainerStarted","Data":"db498c17f60ca0177b5d0ea7de558bb36a6de3f0bb2468f3e2c585d99ae377d2"}
Jul 30 04:43:47.972008 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.971987    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw" event={"ID":"1275fbc0-ef79-4872-8963-65ba5f4587af","Type":"ContainerStarted","Data":"438a5903e7105b6931fc32759ffd967aba6afd710312cd18819faace9820022d"}
Jul 30 04:43:47.972391 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.972103    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"
Jul 30 04:43:47.973669 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.973646    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4" event={"ID":"3c12a586-90ad-49eb-a7f9-d040763f90ad","Type":"ContainerStarted","Data":"732896b4f7f29ae1d9a975b48bfa8fd7008e460c56e226ccf353e539e833d791"}
Jul 30 04:43:47.973795 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.973686    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4" event={"ID":"3c12a586-90ad-49eb-a7f9-d040763f90ad","Type":"ContainerStarted","Data":"60caf2ad0fcb07ce68000c04f852d4466c0def61057020724a077bb74c0442d9"}
Jul 30 04:43:47.973795 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.973705    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"
Jul 30 04:43:47.987867 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:47.987821    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw" podStartSLOduration=0.987803446 podStartE2EDuration="987.803446ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:43:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:43:47.986890299 +0000 UTC m=+3527.025601318" watchObservedRunningTime="2026-07-30 04:43:47.987803446 +0000 UTC m=+3527.026514471"
Jul 30 04:43:48.002757 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:48.002720    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4" podStartSLOduration=1.002707243 podStartE2EDuration="1.002707243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:43:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:43:48.000778761 +0000 UTC m=+3527.039489795" watchObservedRunningTime="2026-07-30 04:43:48.002707243 +0000 UTC m=+3527.041418262"
Jul 30 04:43:58.980392 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:58.980348    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw" podUID="1275fbc0-ef79-4872-8963-65ba5f4587af" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.78:9004: connect: connection refused"
Jul 30 04:43:58.980947 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:43:58.980665    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"
Jul 30 04:44:08.979710 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:44:08.979676    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"
Jul 30 04:44:10.427279 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:44:10.427235    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:44:17.485970 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:44:17.485942    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4_3c12a586-90ad-49eb-a7f9-d040763f90ad/payload-processing/0.log"
Jul 30 04:44:28.033269 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:44:28.033237    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:01.750056 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:01.750029    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:45:01.762781 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:01.762761    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:45:09.474057 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:09.474023    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk_fe21cc92-8356-4828-834c-854b306ef3c0/payload-processing/0.log"
Jul 30 04:45:09.719521 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:09.719494    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4_3c12a586-90ad-49eb-a7f9-d040763f90ad/payload-processing/0.log"
Jul 30 04:45:09.989251 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:09.989226    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:10.296138 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:10.296110    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:10.395173 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:10.395142    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:12.901669 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:12.901611    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:12.998772 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:12.998722    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:13.995964 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:13.995930    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"]
Jul 30 04:45:13.999237 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:13.999216    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:14.001202 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:14.001181    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-1ddcb8-serving-cert\""
Jul 30 04:45:14.010427 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:14.010406    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"]
Jul 30 04:45:14.043721 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:14.043697    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:14.111553 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:14.111508    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk\" (UID: \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:14.111714 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:14.111582    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n2rj\" (UniqueName: \"kubernetes.io/projected/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-kube-api-access-4n2rj\") pod \"maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk\" (UID: \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:14.111714 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:14.111623    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk\" (UID: \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:14.212656 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:14.212628    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n2rj\" (UniqueName: \"kubernetes.io/projected/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-kube-api-access-4n2rj\") pod \"maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk\" (UID: \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:14.212826 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:14.212675    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk\" (UID: \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:14.212826 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:14.212730    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk\" (UID: \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:14.212912 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:45:14.212826    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-serving-cert: secret "maas-api-e2e-ipp-cleanup-1ddcb8-serving-cert" not found
Jul 30 04:45:14.212912 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:45:14.212908    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-maas-api-tls podName:d8bc8f66-bac8-46c9-a3b1-200eb66bbc84 nodeName:}" failed. No retries permitted until 2026-07-30 04:45:14.712884848 +0000 UTC m=+3613.751595845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk" (UID: "d8bc8f66-bac8-46c9-a3b1-200eb66bbc84") : secret "maas-api-e2e-ipp-cleanup-1ddcb8-serving-cert" not found
Jul 30 04:45:14.213384 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:14.213363    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk\" (UID: \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:14.223570 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:14.223548    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n2rj\" (UniqueName: \"kubernetes.io/projected/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-kube-api-access-4n2rj\") pod \"maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk\" (UID: \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:14.716592 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:14.716556    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk\" (UID: \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:14.719151 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:14.719127    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk\" (UID: \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:14.913022 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:14.912984    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:15.052192 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:15.052163    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"]
Jul 30 04:45:15.054111 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:45:15.054081    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8bc8f66_bac8_46c9_a3b1_200eb66bbc84.slice/crio-ca451ed74cc8064a563ae31e1f6f77781aff8572a8810c0b9ca121c6a1c05339 WatchSource:0}: Error finding container ca451ed74cc8064a563ae31e1f6f77781aff8572a8810c0b9ca121c6a1c05339: Status 404 returned error can't find the container with id ca451ed74cc8064a563ae31e1f6f77781aff8572a8810c0b9ca121c6a1c05339
Jul 30 04:45:15.445919 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:15.445877    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk" event={"ID":"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84","Type":"ContainerStarted","Data":"ca451ed74cc8064a563ae31e1f6f77781aff8572a8810c0b9ca121c6a1c05339"}
Jul 30 04:45:15.695442 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:15.695410    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:16.452859 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:16.452824    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk" event={"ID":"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84","Type":"ContainerStarted","Data":"70b7b271c3f8a765a03cf2b2ebeb02991f0cbf4c0fcf08b2a0b50204d6378222"}
Jul 30 04:45:16.453259 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:16.452947    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:16.470497 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:16.470450    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk" podStartSLOduration=3.006018734 podStartE2EDuration="3.470435752s" totalImagesPullingTime="464.417018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:45:16.467753583 +0000 UTC m=+3615.506464604" watchObservedRunningTime="2026-07-30 04:45:16.470435752 +0000 UTC m=+3615.509146770"
Jul 30 04:45:22.463902 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:22.463877    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:37.915339 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:37.915304    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:44.554231 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.554188    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"]
Jul 30 04:45:44.554724 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.554438    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk" podUID="d8bc8f66-bac8-46c9-a3b1-200eb66bbc84" containerName="maas-api" containerID="cri-o://70b7b271c3f8a765a03cf2b2ebeb02991f0cbf4c0fcf08b2a0b50204d6378222" gracePeriod=30
Jul 30 04:45:44.580200 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.580176    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:44.680977 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.680948    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:44.757081 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.757053    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:44.806733 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.806676    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:44.815683 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.815635    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:44.863102 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.863079    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-openshift-service-ca\") pod \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\" (UID: \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\") "
Jul 30 04:45:44.863254 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.863131    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-maas-api-tls\") pod \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\" (UID: \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\") "
Jul 30 04:45:44.863254 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.863181    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4n2rj\" (UniqueName: \"kubernetes.io/projected/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-kube-api-access-4n2rj\") pod \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\" (UID: \"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84\") "
Jul 30 04:45:44.863441 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.863416    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d8bc8f66-bac8-46c9-a3b1-200eb66bbc84" (UID: "d8bc8f66-bac8-46c9-a3b1-200eb66bbc84"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:45:44.865384 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.865361    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-kube-api-access-4n2rj" (OuterVolumeSpecName: "kube-api-access-4n2rj") pod "d8bc8f66-bac8-46c9-a3b1-200eb66bbc84" (UID: "d8bc8f66-bac8-46c9-a3b1-200eb66bbc84"). InnerVolumeSpecName "kube-api-access-4n2rj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:45:44.865456 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.865380    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d8bc8f66-bac8-46c9-a3b1-200eb66bbc84" (UID: "d8bc8f66-bac8-46c9-a3b1-200eb66bbc84"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:45:44.963979 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.963951    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n2rj\" (UniqueName: \"kubernetes.io/projected/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-kube-api-access-4n2rj\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:45:44.963979 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.963976    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-openshift-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:45:44.964153 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:44.963987    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84-maas-api-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:45:45.126963 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:45.126873    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:45.607615 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:45.607582    2569 generic.go:356] "Generic (PLEG): container finished" podID="d8bc8f66-bac8-46c9-a3b1-200eb66bbc84" containerID="70b7b271c3f8a765a03cf2b2ebeb02991f0cbf4c0fcf08b2a0b50204d6378222" exitCode=0
Jul 30 04:45:45.608000 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:45.607625    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk" event={"ID":"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84","Type":"ContainerDied","Data":"70b7b271c3f8a765a03cf2b2ebeb02991f0cbf4c0fcf08b2a0b50204d6378222"}
Jul 30 04:45:45.608000 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:45.607679    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk" event={"ID":"d8bc8f66-bac8-46c9-a3b1-200eb66bbc84","Type":"ContainerDied","Data":"ca451ed74cc8064a563ae31e1f6f77781aff8572a8810c0b9ca121c6a1c05339"}
Jul 30 04:45:45.608000 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:45.607634    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"
Jul 30 04:45:45.608000 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:45.607705    2569 scope.go:117] "RemoveContainer" containerID="70b7b271c3f8a765a03cf2b2ebeb02991f0cbf4c0fcf08b2a0b50204d6378222"
Jul 30 04:45:45.617058 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:45.617042    2569 scope.go:117] "RemoveContainer" containerID="70b7b271c3f8a765a03cf2b2ebeb02991f0cbf4c0fcf08b2a0b50204d6378222"
Jul 30 04:45:45.617280 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:45:45.617261    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70b7b271c3f8a765a03cf2b2ebeb02991f0cbf4c0fcf08b2a0b50204d6378222\": container with ID starting with 70b7b271c3f8a765a03cf2b2ebeb02991f0cbf4c0fcf08b2a0b50204d6378222 not found: ID does not exist" containerID="70b7b271c3f8a765a03cf2b2ebeb02991f0cbf4c0fcf08b2a0b50204d6378222"
Jul 30 04:45:45.617338 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:45.617287    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70b7b271c3f8a765a03cf2b2ebeb02991f0cbf4c0fcf08b2a0b50204d6378222"} err="failed to get container status \"70b7b271c3f8a765a03cf2b2ebeb02991f0cbf4c0fcf08b2a0b50204d6378222\": rpc error: code = NotFound desc = could not find container \"70b7b271c3f8a765a03cf2b2ebeb02991f0cbf4c0fcf08b2a0b50204d6378222\": container with ID starting with 70b7b271c3f8a765a03cf2b2ebeb02991f0cbf4c0fcf08b2a0b50204d6378222 not found: ID does not exist"
Jul 30 04:45:45.623056 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:45.623027    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"]
Jul 30 04:45:45.630118 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:45.630098    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1ddcb8-65b8b6b7fd-6skfk"]
Jul 30 04:45:47.465229 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:47.465190    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8bc8f66-bac8-46c9-a3b1-200eb66bbc84" path="/var/lib/kubelet/pods/d8bc8f66-bac8-46c9-a3b1-200eb66bbc84/volumes"
Jul 30 04:45:51.036383 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:51.036318    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:51.110681 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:51.110657    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:51.305382 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:51.305303    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:45:51.534600 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:45:51.534524    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:46:10.504034 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.504005    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:46:10.527838 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.527806    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"]
Jul 30 04:46:10.528050 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.528026    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk" podUID="fe21cc92-8356-4828-834c-854b306ef3c0" containerName="payload-processing" containerID="cri-o://c8659e01784d536865c56f31fff3c691c9a95d7bf4ddf8d62a097dd6e5dec057" gracePeriod=30
Jul 30 04:46:10.538989 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.538959    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"]
Jul 30 04:46:10.539217 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.539185    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9" podUID="966e3746-6558-4c91-9329-595ce491b633" containerName="payload-pre-processing" containerID="cri-o://fb8b7aac45cf797be6e3f92b5c398ac679f98e8b1c7256c694bd493929b266b3" gracePeriod=30
Jul 30 04:46:10.566979 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.566955    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:46:10.628286 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.628227    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:46:10.685246 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.685213    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:46:10.744645 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.744582    2569 generic.go:356] "Generic (PLEG): container finished" podID="fe21cc92-8356-4828-834c-854b306ef3c0" containerID="c8659e01784d536865c56f31fff3c691c9a95d7bf4ddf8d62a097dd6e5dec057" exitCode=0
Jul 30 04:46:10.744729 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.744654    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk" event={"ID":"fe21cc92-8356-4828-834c-854b306ef3c0","Type":"ContainerDied","Data":"c8659e01784d536865c56f31fff3c691c9a95d7bf4ddf8d62a097dd6e5dec057"}
Jul 30 04:46:10.746553 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.746506    2569 generic.go:356] "Generic (PLEG): container finished" podID="966e3746-6558-4c91-9329-595ce491b633" containerID="fb8b7aac45cf797be6e3f92b5c398ac679f98e8b1c7256c694bd493929b266b3" exitCode=0
Jul 30 04:46:10.746638 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.746584    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9" event={"ID":"966e3746-6558-4c91-9329-595ce491b633","Type":"ContainerDied","Data":"fb8b7aac45cf797be6e3f92b5c398ac679f98e8b1c7256c694bd493929b266b3"}
Jul 30 04:46:10.821154 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.821135    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"
Jul 30 04:46:10.823943 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.823925    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"
Jul 30 04:46:10.873147 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.873092    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jz4g5\" (UniqueName: \"kubernetes.io/projected/fe21cc92-8356-4828-834c-854b306ef3c0-kube-api-access-jz4g5\") pod \"fe21cc92-8356-4828-834c-854b306ef3c0\" (UID: \"fe21cc92-8356-4828-834c-854b306ef3c0\") "
Jul 30 04:46:10.873302 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.873189    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe21cc92-8356-4828-834c-854b306ef3c0-plugins-config-volume\") pod \"fe21cc92-8356-4828-834c-854b306ef3c0\" (UID: \"fe21cc92-8356-4828-834c-854b306ef3c0\") "
Jul 30 04:46:10.873782 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.873755    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe21cc92-8356-4828-834c-854b306ef3c0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fe21cc92-8356-4828-834c-854b306ef3c0" (UID: "fe21cc92-8356-4828-834c-854b306ef3c0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:46:10.875771 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.875741    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe21cc92-8356-4828-834c-854b306ef3c0-kube-api-access-jz4g5" (OuterVolumeSpecName: "kube-api-access-jz4g5") pod "fe21cc92-8356-4828-834c-854b306ef3c0" (UID: "fe21cc92-8356-4828-834c-854b306ef3c0"). InnerVolumeSpecName "kube-api-access-jz4g5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:46:10.974266 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.974235    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/966e3746-6558-4c91-9329-595ce491b633-plugins-config-volume\") pod \"966e3746-6558-4c91-9329-595ce491b633\" (UID: \"966e3746-6558-4c91-9329-595ce491b633\") "
Jul 30 04:46:10.974414 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.974305    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7l5t\" (UniqueName: \"kubernetes.io/projected/966e3746-6558-4c91-9329-595ce491b633-kube-api-access-c7l5t\") pod \"966e3746-6558-4c91-9329-595ce491b633\" (UID: \"966e3746-6558-4c91-9329-595ce491b633\") "
Jul 30 04:46:10.974647 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.974626    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe21cc92-8356-4828-834c-854b306ef3c0-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:46:10.974714 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.974653    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz4g5\" (UniqueName: \"kubernetes.io/projected/fe21cc92-8356-4828-834c-854b306ef3c0-kube-api-access-jz4g5\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:46:10.974714 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.974661    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/966e3746-6558-4c91-9329-595ce491b633-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "966e3746-6558-4c91-9329-595ce491b633" (UID: "966e3746-6558-4c91-9329-595ce491b633"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:46:10.976454 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:10.976432    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/966e3746-6558-4c91-9329-595ce491b633-kube-api-access-c7l5t" (OuterVolumeSpecName: "kube-api-access-c7l5t") pod "966e3746-6558-4c91-9329-595ce491b633" (UID: "966e3746-6558-4c91-9329-595ce491b633"). InnerVolumeSpecName "kube-api-access-c7l5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:46:11.060469 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:11.060393    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:46:11.075680 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:11.075658    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/966e3746-6558-4c91-9329-595ce491b633-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:46:11.075680 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:11.075680    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7l5t\" (UniqueName: \"kubernetes.io/projected/966e3746-6558-4c91-9329-595ce491b633-kube-api-access-c7l5t\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:46:11.752815 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:11.752742    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"
Jul 30 04:46:11.752815 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:11.752745    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9" event={"ID":"966e3746-6558-4c91-9329-595ce491b633","Type":"ContainerDied","Data":"f6dd98b899cfa5c988cd45381580b230bc67604b3a920d8d5f76439a2c693c38"}
Jul 30 04:46:11.753303 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:11.752820    2569 scope.go:117] "RemoveContainer" containerID="fb8b7aac45cf797be6e3f92b5c398ac679f98e8b1c7256c694bd493929b266b3"
Jul 30 04:46:11.754226 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:11.754132    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"
Jul 30 04:46:11.754226 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:11.754156    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk" event={"ID":"fe21cc92-8356-4828-834c-854b306ef3c0","Type":"ContainerDied","Data":"30f911ee3a63850a03f0c2f8eebbeaa12a354b8b296c41bc264035cd71730bc7"}
Jul 30 04:46:11.762292 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:11.762274    2569 scope.go:117] "RemoveContainer" containerID="c8659e01784d536865c56f31fff3c691c9a95d7bf4ddf8d62a097dd6e5dec057"
Jul 30 04:46:11.771846 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:11.771573    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"]
Jul 30 04:46:11.773988 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:11.773969    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a1808a-574bb5755b-d54jk"]
Jul 30 04:46:11.783832 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:11.783807    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"]
Jul 30 04:46:11.788006 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:11.787985    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a1808a-6446668d8-zd4n9"]
Jul 30 04:46:13.465980 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:13.465951    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="966e3746-6558-4c91-9329-595ce491b633" path="/var/lib/kubelet/pods/966e3746-6558-4c91-9329-595ce491b633/volumes"
Jul 30 04:46:13.466358 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:13.466295    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe21cc92-8356-4828-834c-854b306ef3c0" path="/var/lib/kubelet/pods/fe21cc92-8356-4828-834c-854b306ef3c0/volumes"
Jul 30 04:46:16.091368 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:16.091331    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:46:29.063991 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:29.063908    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:46:52.696196 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:52.696167    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:46:53.012283 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:53.012194    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:46:58.439090 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:58.439056    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:46:58.509734 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:58.509706    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:46:58.709726 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:58.709650    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:46:58.942527 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:46:58.942492    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:09.518547 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.518496    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:09.548095 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.548055    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"]
Jul 30 04:47:09.548365 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.548337    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4" podUID="3c12a586-90ad-49eb-a7f9-d040763f90ad" containerName="payload-processing" containerID="cri-o://732896b4f7f29ae1d9a975b48bfa8fd7008e460c56e226ccf353e539e833d791" gracePeriod=30
Jul 30 04:47:09.556421 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.556391    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"]
Jul 30 04:47:09.556701 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.556637    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw" podUID="1275fbc0-ef79-4872-8963-65ba5f4587af" containerName="payload-pre-processing" containerID="cri-o://db498c17f60ca0177b5d0ea7de558bb36a6de3f0bb2468f3e2c585d99ae377d2" gracePeriod=30
Jul 30 04:47:09.585275 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.585245    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:09.685815 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.685779    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:09.740568 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.740520    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:09.825733 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.825712    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"
Jul 30 04:47:09.828463 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.828444    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"
Jul 30 04:47:09.944909 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.944878    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1275fbc0-ef79-4872-8963-65ba5f4587af-plugins-config-volume\") pod \"1275fbc0-ef79-4872-8963-65ba5f4587af\" (UID: \"1275fbc0-ef79-4872-8963-65ba5f4587af\") "
Jul 30 04:47:09.945065 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.944927    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxhlf\" (UniqueName: \"kubernetes.io/projected/3c12a586-90ad-49eb-a7f9-d040763f90ad-kube-api-access-qxhlf\") pod \"3c12a586-90ad-49eb-a7f9-d040763f90ad\" (UID: \"3c12a586-90ad-49eb-a7f9-d040763f90ad\") "
Jul 30 04:47:09.945065 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.944965    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c12a586-90ad-49eb-a7f9-d040763f90ad-plugins-config-volume\") pod \"3c12a586-90ad-49eb-a7f9-d040763f90ad\" (UID: \"3c12a586-90ad-49eb-a7f9-d040763f90ad\") "
Jul 30 04:47:09.945065 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.945015    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vz7d4\" (UniqueName: \"kubernetes.io/projected/1275fbc0-ef79-4872-8963-65ba5f4587af-kube-api-access-vz7d4\") pod \"1275fbc0-ef79-4872-8963-65ba5f4587af\" (UID: \"1275fbc0-ef79-4872-8963-65ba5f4587af\") "
Jul 30 04:47:09.945325 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.945300    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1275fbc0-ef79-4872-8963-65ba5f4587af-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1275fbc0-ef79-4872-8963-65ba5f4587af" (UID: "1275fbc0-ef79-4872-8963-65ba5f4587af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:47:09.945373 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.945327    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c12a586-90ad-49eb-a7f9-d040763f90ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3c12a586-90ad-49eb-a7f9-d040763f90ad" (UID: "3c12a586-90ad-49eb-a7f9-d040763f90ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:47:09.947256 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.947231    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c12a586-90ad-49eb-a7f9-d040763f90ad-kube-api-access-qxhlf" (OuterVolumeSpecName: "kube-api-access-qxhlf") pod "3c12a586-90ad-49eb-a7f9-d040763f90ad" (UID: "3c12a586-90ad-49eb-a7f9-d040763f90ad"). InnerVolumeSpecName "kube-api-access-qxhlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:47:09.947362 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:09.947262    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1275fbc0-ef79-4872-8963-65ba5f4587af-kube-api-access-vz7d4" (OuterVolumeSpecName: "kube-api-access-vz7d4") pod "1275fbc0-ef79-4872-8963-65ba5f4587af" (UID: "1275fbc0-ef79-4872-8963-65ba5f4587af"). InnerVolumeSpecName "kube-api-access-vz7d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:47:10.046566 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.046520    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz7d4\" (UniqueName: \"kubernetes.io/projected/1275fbc0-ef79-4872-8963-65ba5f4587af-kube-api-access-vz7d4\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:47:10.046566 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.046563    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1275fbc0-ef79-4872-8963-65ba5f4587af-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:47:10.046707 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.046573    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxhlf\" (UniqueName: \"kubernetes.io/projected/3c12a586-90ad-49eb-a7f9-d040763f90ad-kube-api-access-qxhlf\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:47:10.046707 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.046582    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c12a586-90ad-49eb-a7f9-d040763f90ad-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:47:10.058166 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.058103    2569 generic.go:356] "Generic (PLEG): container finished" podID="3c12a586-90ad-49eb-a7f9-d040763f90ad" containerID="732896b4f7f29ae1d9a975b48bfa8fd7008e460c56e226ccf353e539e833d791" exitCode=0
Jul 30 04:47:10.058166 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.058146    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"
Jul 30 04:47:10.058324 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.058191    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4" event={"ID":"3c12a586-90ad-49eb-a7f9-d040763f90ad","Type":"ContainerDied","Data":"732896b4f7f29ae1d9a975b48bfa8fd7008e460c56e226ccf353e539e833d791"}
Jul 30 04:47:10.058324 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.058239    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4" event={"ID":"3c12a586-90ad-49eb-a7f9-d040763f90ad","Type":"ContainerDied","Data":"60caf2ad0fcb07ce68000c04f852d4466c0def61057020724a077bb74c0442d9"}
Jul 30 04:47:10.058324 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.058273    2569 scope.go:117] "RemoveContainer" containerID="732896b4f7f29ae1d9a975b48bfa8fd7008e460c56e226ccf353e539e833d791"
Jul 30 04:47:10.059598 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.059577    2569 generic.go:356] "Generic (PLEG): container finished" podID="1275fbc0-ef79-4872-8963-65ba5f4587af" containerID="db498c17f60ca0177b5d0ea7de558bb36a6de3f0bb2468f3e2c585d99ae377d2" exitCode=0
Jul 30 04:47:10.059717 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.059615    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"
Jul 30 04:47:10.059717 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.059642    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw" event={"ID":"1275fbc0-ef79-4872-8963-65ba5f4587af","Type":"ContainerDied","Data":"db498c17f60ca0177b5d0ea7de558bb36a6de3f0bb2468f3e2c585d99ae377d2"}
Jul 30 04:47:10.059717 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.059672    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw" event={"ID":"1275fbc0-ef79-4872-8963-65ba5f4587af","Type":"ContainerDied","Data":"438a5903e7105b6931fc32759ffd967aba6afd710312cd18819faace9820022d"}
Jul 30 04:47:10.072731 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.072715    2569 scope.go:117] "RemoveContainer" containerID="732896b4f7f29ae1d9a975b48bfa8fd7008e460c56e226ccf353e539e833d791"
Jul 30 04:47:10.073012 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:47:10.072994    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"732896b4f7f29ae1d9a975b48bfa8fd7008e460c56e226ccf353e539e833d791\": container with ID starting with 732896b4f7f29ae1d9a975b48bfa8fd7008e460c56e226ccf353e539e833d791 not found: ID does not exist" containerID="732896b4f7f29ae1d9a975b48bfa8fd7008e460c56e226ccf353e539e833d791"
Jul 30 04:47:10.073075 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.073019    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"732896b4f7f29ae1d9a975b48bfa8fd7008e460c56e226ccf353e539e833d791"} err="failed to get container status \"732896b4f7f29ae1d9a975b48bfa8fd7008e460c56e226ccf353e539e833d791\": rpc error: code = NotFound desc = could not find container \"732896b4f7f29ae1d9a975b48bfa8fd7008e460c56e226ccf353e539e833d791\": container with ID starting with 732896b4f7f29ae1d9a975b48bfa8fd7008e460c56e226ccf353e539e833d791 not found: ID does not exist"
Jul 30 04:47:10.073075 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.073035    2569 scope.go:117] "RemoveContainer" containerID="db498c17f60ca0177b5d0ea7de558bb36a6de3f0bb2468f3e2c585d99ae377d2"
Jul 30 04:47:10.081897 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.081878    2569 scope.go:117] "RemoveContainer" containerID="db498c17f60ca0177b5d0ea7de558bb36a6de3f0bb2468f3e2c585d99ae377d2"
Jul 30 04:47:10.082130 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:47:10.082107    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db498c17f60ca0177b5d0ea7de558bb36a6de3f0bb2468f3e2c585d99ae377d2\": container with ID starting with db498c17f60ca0177b5d0ea7de558bb36a6de3f0bb2468f3e2c585d99ae377d2 not found: ID does not exist" containerID="db498c17f60ca0177b5d0ea7de558bb36a6de3f0bb2468f3e2c585d99ae377d2"
Jul 30 04:47:10.082203 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.082139    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"]
Jul 30 04:47:10.082203 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.082139    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db498c17f60ca0177b5d0ea7de558bb36a6de3f0bb2468f3e2c585d99ae377d2"} err="failed to get container status \"db498c17f60ca0177b5d0ea7de558bb36a6de3f0bb2468f3e2c585d99ae377d2\": rpc error: code = NotFound desc = could not find container \"db498c17f60ca0177b5d0ea7de558bb36a6de3f0bb2468f3e2c585d99ae377d2\": container with ID starting with db498c17f60ca0177b5d0ea7de558bb36a6de3f0bb2468f3e2c585d99ae377d2 not found: ID does not exist"
Jul 30 04:47:10.085941 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.085920    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-eda6fc-5f74ff885d-7dhv4"]
Jul 30 04:47:10.093266 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.093249    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:10.094481 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.094463    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"]
Jul 30 04:47:10.100383 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:10.100358    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-eda6fc-687786d8b8-5zfkw"]
Jul 30 04:47:11.465294 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:11.465254    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1275fbc0-ef79-4872-8963-65ba5f4587af" path="/var/lib/kubelet/pods/1275fbc0-ef79-4872-8963-65ba5f4587af/volumes"
Jul 30 04:47:11.465808 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:11.465768    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c12a586-90ad-49eb-a7f9-d040763f90ad" path="/var/lib/kubelet/pods/3c12a586-90ad-49eb-a7f9-d040763f90ad/volumes"
Jul 30 04:47:15.878408 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:15.878370    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:15.979425 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:15.979388    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:16.150681 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:16.150602    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:16.383597 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:16.383558    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:26.941062 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:26.941025    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"]
Jul 30 04:47:26.941554 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:26.941286    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8" podUID="e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5" containerName="maas-api" containerID="cri-o://0502cbffa4493dd74cc2df5ba8b8169444d6851ffe8af5a48c334736ab1f61ff" gracePeriod=30
Jul 30 04:47:26.969872 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:26.969844    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:27.114084 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.114050    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:27.150489 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.150449    2569 generic.go:356] "Generic (PLEG): container finished" podID="e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5" containerID="0502cbffa4493dd74cc2df5ba8b8169444d6851ffe8af5a48c334736ab1f61ff" exitCode=0
Jul 30 04:47:27.150684 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.150551    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8" event={"ID":"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5","Type":"ContainerDied","Data":"0502cbffa4493dd74cc2df5ba8b8169444d6851ffe8af5a48c334736ab1f61ff"}
Jul 30 04:47:27.170271 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.170242    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:27.197365 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.197313    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:47:27.242073 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.242041    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:27.289088 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.289065    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsjx6\" (UniqueName: \"kubernetes.io/projected/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-kube-api-access-bsjx6\") pod \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\" (UID: \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\") "
Jul 30 04:47:27.289208 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.289129    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-maas-api-tls\") pod \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\" (UID: \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\") "
Jul 30 04:47:27.289208 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.289200    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-openshift-service-ca\") pod \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\" (UID: \"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5\") "
Jul 30 04:47:27.289583 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.289526    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5" (UID: "e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:47:27.291273 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.291255    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-kube-api-access-bsjx6" (OuterVolumeSpecName: "kube-api-access-bsjx6") pod "e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5" (UID: "e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5"). InnerVolumeSpecName "kube-api-access-bsjx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:47:27.291347 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.291309    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5" (UID: "e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:47:27.389972 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.389942    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsjx6\" (UniqueName: \"kubernetes.io/projected/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-kube-api-access-bsjx6\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:47:27.389972 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.389969    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-maas-api-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:47:27.390101 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.389983    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5-openshift-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:47:27.605977 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:27.605954    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:28.157610 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:28.157584    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"
Jul 30 04:47:28.157610 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:28.157603    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8" event={"ID":"e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5","Type":"ContainerDied","Data":"d291e38a003f35dcbdc14402aea8f81e225251007ba8e31412b291e7aef826e2"}
Jul 30 04:47:28.158078 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:28.157660    2569 scope.go:117] "RemoveContainer" containerID="0502cbffa4493dd74cc2df5ba8b8169444d6851ffe8af5a48c334736ab1f61ff"
Jul 30 04:47:28.172772 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:28.172749    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"]
Jul 30 04:47:28.176798 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:28.176779    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-324a29-5f8545f766-bzgq8"]
Jul 30 04:47:29.465124 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:29.465087    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5" path="/var/lib/kubelet/pods/e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5/volumes"
Jul 30 04:47:33.473906 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:33.473869    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:33.573930 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:33.573903    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:33.744383 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:33.744305    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:33.966700 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:33.966670    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:44.486691 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.486653    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"]
Jul 30 04:47:44.487129 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.486925    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7" podUID="7cc53b5a-d515-4c21-b7bd-fa28d8c08f84" containerName="maas-api" containerID="cri-o://a56773a753f1ae3fae382da711d95962bcd41619cd840a05a5205c02286695ab" gracePeriod=30
Jul 30 04:47:44.510905 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.510879    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:44.544055 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.544031    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"]
Jul 30 04:47:44.544266 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.544221    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj" podUID="bd46d82e-52b7-4b2e-882d-edbc97582a1d" containerName="payload-processing" containerID="cri-o://0cce40f9eeb4bc53b2832f1865fa57c08fc23898c912ea0436f16cadf763b8f9" gracePeriod=30
Jul 30 04:47:44.556718 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.556688    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"]
Jul 30 04:47:44.556911 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.556881    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q" podUID="ecc7bb6a-6fa6-4237-84f7-030136d77069" containerName="payload-pre-processing" containerID="cri-o://d87f70ffa6dbe3c945ea9e42b316e4719ae0d06cf617d15fe512bb2bbf203d56" gracePeriod=30
Jul 30 04:47:44.579428 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.579406    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:44.634877 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.634854    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:44.698818 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.698786    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:44.828333 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.828310    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"
Jul 30 04:47:44.834499 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.834467    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:47:44.846398 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.846309    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"
Jul 30 04:47:44.940596 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.940567    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rjlw\" (UniqueName: \"kubernetes.io/projected/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-kube-api-access-7rjlw\") pod \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\" (UID: \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\") "
Jul 30 04:47:44.940779 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.940602    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-openshift-service-ca\") pod \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\" (UID: \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\") "
Jul 30 04:47:44.940779 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.940622    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8np4\" (UniqueName: \"kubernetes.io/projected/bd46d82e-52b7-4b2e-882d-edbc97582a1d-kube-api-access-w8np4\") pod \"bd46d82e-52b7-4b2e-882d-edbc97582a1d\" (UID: \"bd46d82e-52b7-4b2e-882d-edbc97582a1d\") "
Jul 30 04:47:44.940779 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.940647    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecc7bb6a-6fa6-4237-84f7-030136d77069-plugins-config-volume\") pod \"ecc7bb6a-6fa6-4237-84f7-030136d77069\" (UID: \"ecc7bb6a-6fa6-4237-84f7-030136d77069\") "
Jul 30 04:47:44.940779 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.940704    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xqnt\" (UniqueName: \"kubernetes.io/projected/ecc7bb6a-6fa6-4237-84f7-030136d77069-kube-api-access-9xqnt\") pod \"ecc7bb6a-6fa6-4237-84f7-030136d77069\" (UID: \"ecc7bb6a-6fa6-4237-84f7-030136d77069\") "
Jul 30 04:47:44.940779 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.940743    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-maas-api-tls\") pod \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\" (UID: \"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84\") "
Jul 30 04:47:44.940779 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.940779    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd46d82e-52b7-4b2e-882d-edbc97582a1d-plugins-config-volume\") pod \"bd46d82e-52b7-4b2e-882d-edbc97582a1d\" (UID: \"bd46d82e-52b7-4b2e-882d-edbc97582a1d\") "
Jul 30 04:47:44.941117 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.940991    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7cc53b5a-d515-4c21-b7bd-fa28d8c08f84" (UID: "7cc53b5a-d515-4c21-b7bd-fa28d8c08f84"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:47:44.941117 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.941028    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecc7bb6a-6fa6-4237-84f7-030136d77069-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ecc7bb6a-6fa6-4237-84f7-030136d77069" (UID: "ecc7bb6a-6fa6-4237-84f7-030136d77069"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:47:44.941590 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.941563    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd46d82e-52b7-4b2e-882d-edbc97582a1d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bd46d82e-52b7-4b2e-882d-edbc97582a1d" (UID: "bd46d82e-52b7-4b2e-882d-edbc97582a1d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:47:44.942982 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.942955    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7cc53b5a-d515-4c21-b7bd-fa28d8c08f84" (UID: "7cc53b5a-d515-4c21-b7bd-fa28d8c08f84"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:47:44.943079 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.943017    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd46d82e-52b7-4b2e-882d-edbc97582a1d-kube-api-access-w8np4" (OuterVolumeSpecName: "kube-api-access-w8np4") pod "bd46d82e-52b7-4b2e-882d-edbc97582a1d" (UID: "bd46d82e-52b7-4b2e-882d-edbc97582a1d"). InnerVolumeSpecName "kube-api-access-w8np4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:47:44.943079 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.943030    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecc7bb6a-6fa6-4237-84f7-030136d77069-kube-api-access-9xqnt" (OuterVolumeSpecName: "kube-api-access-9xqnt") pod "ecc7bb6a-6fa6-4237-84f7-030136d77069" (UID: "ecc7bb6a-6fa6-4237-84f7-030136d77069"). InnerVolumeSpecName "kube-api-access-9xqnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:47:44.943079 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:44.943044    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-kube-api-access-7rjlw" (OuterVolumeSpecName: "kube-api-access-7rjlw") pod "7cc53b5a-d515-4c21-b7bd-fa28d8c08f84" (UID: "7cc53b5a-d515-4c21-b7bd-fa28d8c08f84"). InnerVolumeSpecName "kube-api-access-7rjlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:47:45.042290 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.042264    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-maas-api-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:47:45.042290 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.042287    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd46d82e-52b7-4b2e-882d-edbc97582a1d-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:47:45.042453 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.042298    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rjlw\" (UniqueName: \"kubernetes.io/projected/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-kube-api-access-7rjlw\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:47:45.042453 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.042307    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84-openshift-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:47:45.042453 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.042316    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8np4\" (UniqueName: \"kubernetes.io/projected/bd46d82e-52b7-4b2e-882d-edbc97582a1d-kube-api-access-w8np4\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:47:45.042453 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.042325    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecc7bb6a-6fa6-4237-84f7-030136d77069-plugins-config-volume\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:47:45.042453 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.042334    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xqnt\" (UniqueName: \"kubernetes.io/projected/ecc7bb6a-6fa6-4237-84f7-030136d77069-kube-api-access-9xqnt\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\""
Jul 30 04:47:45.071886 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.071858    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:45.250154 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.250110    2569 generic.go:356] "Generic (PLEG): container finished" podID="ecc7bb6a-6fa6-4237-84f7-030136d77069" containerID="d87f70ffa6dbe3c945ea9e42b316e4719ae0d06cf617d15fe512bb2bbf203d56" exitCode=0
Jul 30 04:47:45.250154 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.250156    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"
Jul 30 04:47:45.250382 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.250212    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q" event={"ID":"ecc7bb6a-6fa6-4237-84f7-030136d77069","Type":"ContainerDied","Data":"d87f70ffa6dbe3c945ea9e42b316e4719ae0d06cf617d15fe512bb2bbf203d56"}
Jul 30 04:47:45.250382 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.250261    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q" event={"ID":"ecc7bb6a-6fa6-4237-84f7-030136d77069","Type":"ContainerDied","Data":"28a0200ac841bbbe16d11a6f98015ab0a6fa86265c5e3eed79ae84662e0fb98e"}
Jul 30 04:47:45.250382 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.250282    2569 scope.go:117] "RemoveContainer" containerID="d87f70ffa6dbe3c945ea9e42b316e4719ae0d06cf617d15fe512bb2bbf203d56"
Jul 30 04:47:45.251660 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.251640    2569 generic.go:356] "Generic (PLEG): container finished" podID="bd46d82e-52b7-4b2e-882d-edbc97582a1d" containerID="0cce40f9eeb4bc53b2832f1865fa57c08fc23898c912ea0436f16cadf763b8f9" exitCode=0
Jul 30 04:47:45.251948 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.251669    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj" event={"ID":"bd46d82e-52b7-4b2e-882d-edbc97582a1d","Type":"ContainerDied","Data":"0cce40f9eeb4bc53b2832f1865fa57c08fc23898c912ea0436f16cadf763b8f9"}
Jul 30 04:47:45.251948 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.251699    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"
Jul 30 04:47:45.251948 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.251718    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj" event={"ID":"bd46d82e-52b7-4b2e-882d-edbc97582a1d","Type":"ContainerDied","Data":"f0aec396e8f8a7a69ffcbdbba4b9533d1b063bb7c8463bf7568df283d8289467"}
Jul 30 04:47:45.253346 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.253314    2569 generic.go:356] "Generic (PLEG): container finished" podID="7cc53b5a-d515-4c21-b7bd-fa28d8c08f84" containerID="a56773a753f1ae3fae382da711d95962bcd41619cd840a05a5205c02286695ab" exitCode=0
Jul 30 04:47:45.253459 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.253356    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"
Jul 30 04:47:45.253459 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.253356    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7" event={"ID":"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84","Type":"ContainerDied","Data":"a56773a753f1ae3fae382da711d95962bcd41619cd840a05a5205c02286695ab"}
Jul 30 04:47:45.253579 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.253452    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7" event={"ID":"7cc53b5a-d515-4c21-b7bd-fa28d8c08f84","Type":"ContainerDied","Data":"7a16f0f46354041d0e7726d2cd0170c9015faf077bd5f8ef33bf6e0c1dd6273e"}
Jul 30 04:47:45.259811 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.259788    2569 scope.go:117] "RemoveContainer" containerID="d87f70ffa6dbe3c945ea9e42b316e4719ae0d06cf617d15fe512bb2bbf203d56"
Jul 30 04:47:45.260095 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:47:45.260065    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d87f70ffa6dbe3c945ea9e42b316e4719ae0d06cf617d15fe512bb2bbf203d56\": container with ID starting with d87f70ffa6dbe3c945ea9e42b316e4719ae0d06cf617d15fe512bb2bbf203d56 not found: ID does not exist" containerID="d87f70ffa6dbe3c945ea9e42b316e4719ae0d06cf617d15fe512bb2bbf203d56"
Jul 30 04:47:45.260182 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.260106    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d87f70ffa6dbe3c945ea9e42b316e4719ae0d06cf617d15fe512bb2bbf203d56"} err="failed to get container status \"d87f70ffa6dbe3c945ea9e42b316e4719ae0d06cf617d15fe512bb2bbf203d56\": rpc error: code = NotFound desc = could not find container \"d87f70ffa6dbe3c945ea9e42b316e4719ae0d06cf617d15fe512bb2bbf203d56\": container with ID starting with d87f70ffa6dbe3c945ea9e42b316e4719ae0d06cf617d15fe512bb2bbf203d56 not found: ID does not exist"
Jul 30 04:47:45.260182 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.260131    2569 scope.go:117] "RemoveContainer" containerID="0cce40f9eeb4bc53b2832f1865fa57c08fc23898c912ea0436f16cadf763b8f9"
Jul 30 04:47:45.268972 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.268955    2569 scope.go:117] "RemoveContainer" containerID="0cce40f9eeb4bc53b2832f1865fa57c08fc23898c912ea0436f16cadf763b8f9"
Jul 30 04:47:45.269215 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:47:45.269195    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cce40f9eeb4bc53b2832f1865fa57c08fc23898c912ea0436f16cadf763b8f9\": container with ID starting with 0cce40f9eeb4bc53b2832f1865fa57c08fc23898c912ea0436f16cadf763b8f9 not found: ID does not exist" containerID="0cce40f9eeb4bc53b2832f1865fa57c08fc23898c912ea0436f16cadf763b8f9"
Jul 30 04:47:45.269262 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.269222    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cce40f9eeb4bc53b2832f1865fa57c08fc23898c912ea0436f16cadf763b8f9"} err="failed to get container status \"0cce40f9eeb4bc53b2832f1865fa57c08fc23898c912ea0436f16cadf763b8f9\": rpc error: code = NotFound desc = could not find container \"0cce40f9eeb4bc53b2832f1865fa57c08fc23898c912ea0436f16cadf763b8f9\": container with ID starting with 0cce40f9eeb4bc53b2832f1865fa57c08fc23898c912ea0436f16cadf763b8f9 not found: ID does not exist"
Jul 30 04:47:45.269262 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.269239    2569 scope.go:117] "RemoveContainer" containerID="a56773a753f1ae3fae382da711d95962bcd41619cd840a05a5205c02286695ab"
Jul 30 04:47:45.276999 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.276983    2569 scope.go:117] "RemoveContainer" containerID="a56773a753f1ae3fae382da711d95962bcd41619cd840a05a5205c02286695ab"
Jul 30 04:47:45.277947 ip-10-0-136-104 kubenswrapper[2569]: E0730 04:47:45.277916    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a56773a753f1ae3fae382da711d95962bcd41619cd840a05a5205c02286695ab\": container with ID starting with a56773a753f1ae3fae382da711d95962bcd41619cd840a05a5205c02286695ab not found: ID does not exist" containerID="a56773a753f1ae3fae382da711d95962bcd41619cd840a05a5205c02286695ab"
Jul 30 04:47:45.277947 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.277941    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a56773a753f1ae3fae382da711d95962bcd41619cd840a05a5205c02286695ab"} err="failed to get container status \"a56773a753f1ae3fae382da711d95962bcd41619cd840a05a5205c02286695ab\": rpc error: code = NotFound desc = could not find container \"a56773a753f1ae3fae382da711d95962bcd41619cd840a05a5205c02286695ab\": container with ID starting with a56773a753f1ae3fae382da711d95962bcd41619cd840a05a5205c02286695ab not found: ID does not exist"
Jul 30 04:47:45.279278 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.279261    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"]
Jul 30 04:47:45.283213 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.283191    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-64af9f-585796496-9w86q"]
Jul 30 04:47:45.293167 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.293117    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"]
Jul 30 04:47:45.296659 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.296641    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-64af9f-897bb4574-mvcpj"]
Jul 30 04:47:45.306604 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.306584    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"]
Jul 30 04:47:45.311085 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.311065    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-64af9f-65b8ff79dd-glkg7"]
Jul 30 04:47:45.466817 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.466785    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cc53b5a-d515-4c21-b7bd-fa28d8c08f84" path="/var/lib/kubelet/pods/7cc53b5a-d515-4c21-b7bd-fa28d8c08f84/volumes"
Jul 30 04:47:45.467127 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.467114    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd46d82e-52b7-4b2e-882d-edbc97582a1d" path="/var/lib/kubelet/pods/bd46d82e-52b7-4b2e-882d-edbc97582a1d/volumes"
Jul 30 04:47:45.467416 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:45.467405    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecc7bb6a-6fa6-4237-84f7-030136d77069" path="/var/lib/kubelet/pods/ecc7bb6a-6fa6-4237-84f7-030136d77069/volumes"
Jul 30 04:47:50.905276 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:50.905241    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:50.963467 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:50.963440    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:51.162422 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:51.162337    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:51.383830 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:51.383801    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:47:57.792284 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:57.792214    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-599789d45f-g7rh7_2eed90ee-243a-45b0-8a41-b13a62f27d0f/copy-manifests/0.log"
Jul 30 04:47:57.805357 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:57.805313    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-599789d45f-g7rh7_2eed90ee-243a-45b0-8a41-b13a62f27d0f/manager/0.log"
Jul 30 04:47:58.413764 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:58.413721    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-86659d4467-qf4jr_4689f77d-2389-4dc5-bae4-427459c63d82/maas-api/0.log"
Jul 30 04:47:58.869344 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:58.869321    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-bpzjd_6b0117bb-3d81-4a0b-bc50-c72fd1585c99/postgres/0.log"
Jul 30 04:47:59.654636 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:59.654608    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg_9e1d39de-2961-4e11-b991-f2828bbae826/util/0.log"
Jul 30 04:47:59.664960 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:59.664932    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg_9e1d39de-2961-4e11-b991-f2828bbae826/pull/0.log"
Jul 30 04:47:59.674680 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:59.674663    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg_9e1d39de-2961-4e11-b991-f2828bbae826/extract/0.log"
Jul 30 04:47:59.789830 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:59.789806    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg_5ae07334-cac6-46e3-a973-bc5eae480e07/extract/0.log"
Jul 30 04:47:59.795873 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:59.795845    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg_5ae07334-cac6-46e3-a973-bc5eae480e07/util/0.log"
Jul 30 04:47:59.802016 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:59.801999    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg_5ae07334-cac6-46e3-a973-bc5eae480e07/pull/0.log"
Jul 30 04:47:59.920724 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:59.920668    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv_40e22250-90e5-451e-b5ba-cba344f96343/util/0.log"
Jul 30 04:47:59.926894 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:59.926878    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv_40e22250-90e5-451e-b5ba-cba344f96343/pull/0.log"
Jul 30 04:47:59.933152 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:47:59.933134    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv_40e22250-90e5-451e-b5ba-cba344f96343/extract/0.log"
Jul 30 04:48:00.045067 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:00.045041    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr_08a1152d-bee4-464f-b723-c71b08f6753c/util/0.log"
Jul 30 04:48:00.051620 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:00.051592    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr_08a1152d-bee4-464f-b723-c71b08f6753c/pull/0.log"
Jul 30 04:48:00.057740 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:00.057724    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr_08a1152d-bee4-464f-b723-c71b08f6753c/extract/0.log"
Jul 30 04:48:01.590122 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:01.590097    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58ccc58984-75hc6_8dd4c125-2623-4e9e-a868-f4ce0157d66a/kube-auth-proxy/0.log"
Jul 30 04:48:01.943060 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:01.942991    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4b7fb_41ac76d3-11aa-4d0d-a44e-d10297c4d5cd/payload-pre-processing/0.log"
Jul 30 04:48:02.527586 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:02.527520    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-d4mt4_751ae2c6-4e75-46db-ba51-8e859c550f48/main/0.log"
Jul 30 04:48:02.881617 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:02.881521    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f77j5f_226f73cc-89c7-41ac-ae89-c5fbc12149e3/main/0.log"
Jul 30 04:48:03.002344 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:03.002315    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-xqlkp_28cab731-6be8-4ec4-b837-4a662b87ef13/main/0.log"
Jul 30 04:48:06.426304 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:06.426269    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:48:06.475990 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:06.475960    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:48:08.274610 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:08.274577    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:48:08.370757 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:08.370726    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log" limit=1024
Jul 30 04:48:11.085195 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:11.085168    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vxddq_a26002ce-a680-4af4-9513-044a00d2393d/global-pull-secret-syncer/0.log"
Jul 30 04:48:11.175504 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:11.175476    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fxtxs_80d2e732-5a45-4f32-bec1-768138b5d8ae/konnectivity-agent/0.log"
Jul 30 04:48:11.288580 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:11.288555    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-104.ec2.internal_9ae53f74fb865d246d654758e9704219/haproxy/0.log"
Jul 30 04:48:14.932079 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:14.932041    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg_9e1d39de-2961-4e11-b991-f2828bbae826/extract/0.log"
Jul 30 04:48:14.956334 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:14.956306    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg_9e1d39de-2961-4e11-b991-f2828bbae826/util/0.log"
Jul 30 04:48:14.984307 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:14.984288    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759484dg_9e1d39de-2961-4e11-b991-f2828bbae826/pull/0.log"
Jul 30 04:48:15.039829 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:15.039810    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg_5ae07334-cac6-46e3-a973-bc5eae480e07/extract/0.log"
Jul 30 04:48:15.068308 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:15.068283    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg_5ae07334-cac6-46e3-a973-bc5eae480e07/util/0.log"
Jul 30 04:48:15.094970 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:15.094945    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sfgcg_5ae07334-cac6-46e3-a973-bc5eae480e07/pull/0.log"
Jul 30 04:48:15.128771 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:15.128751    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv_40e22250-90e5-451e-b5ba-cba344f96343/extract/0.log"
Jul 30 04:48:15.155343 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:15.155320    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv_40e22250-90e5-451e-b5ba-cba344f96343/util/0.log"
Jul 30 04:48:15.183360 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:15.183297    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t9wnv_40e22250-90e5-451e-b5ba-cba344f96343/pull/0.log"
Jul 30 04:48:15.212422 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:15.212402    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr_08a1152d-bee4-464f-b723-c71b08f6753c/extract/0.log"
Jul 30 04:48:15.235603 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:15.235585    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr_08a1152d-bee4-464f-b723-c71b08f6753c/util/0.log"
Jul 30 04:48:15.257093 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:15.257074    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gxvqr_08a1152d-bee4-464f-b723-c71b08f6753c/pull/0.log"
Jul 30 04:48:17.745421 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:17.745392    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ltv96_e6c5fbeb-b997-4d55-b2bc-9eeed9a36fa7/cluster-monitoring-operator/0.log"
Jul 30 04:48:18.066522 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:18.066498    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qd6jz_04b2f1f3-f80c-4137-973d-e789d2c488ce/node-exporter/0.log"
Jul 30 04:48:18.094197 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:18.094174    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qd6jz_04b2f1f3-f80c-4137-973d-e789d2c488ce/kube-rbac-proxy/0.log"
Jul 30 04:48:18.117686 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:18.117663    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qd6jz_04b2f1f3-f80c-4137-973d-e789d2c488ce/init-textfile/0.log"
Jul 30 04:48:20.043590 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:20.043563    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-tcpmc_56a7549c-0cff-4485-8518-30d431954079/networking-console-plugin/0.log"
Jul 30 04:48:20.619258 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:20.619224    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/2.log"
Jul 30 04:48:20.623014 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:20.622991    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-tth9v_30f6ab6d-dc60-40c5-b6b9-9c22afcaf8e0/console-operator/3.log"
Jul 30 04:48:21.038202 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038167    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"]
Jul 30 04:48:21.038592 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038575    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c12a586-90ad-49eb-a7f9-d040763f90ad" containerName="payload-processing"
Jul 30 04:48:21.038695 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038594    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c12a586-90ad-49eb-a7f9-d040763f90ad" containerName="payload-processing"
Jul 30 04:48:21.038695 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038608    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1275fbc0-ef79-4872-8963-65ba5f4587af" containerName="payload-pre-processing"
Jul 30 04:48:21.038695 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038616    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1275fbc0-ef79-4872-8963-65ba5f4587af" containerName="payload-pre-processing"
Jul 30 04:48:21.038695 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038630    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5" containerName="maas-api"
Jul 30 04:48:21.038695 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038638    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5" containerName="maas-api"
Jul 30 04:48:21.038695 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038655    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cc53b5a-d515-4c21-b7bd-fa28d8c08f84" containerName="maas-api"
Jul 30 04:48:21.038695 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038663    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cc53b5a-d515-4c21-b7bd-fa28d8c08f84" containerName="maas-api"
Jul 30 04:48:21.038695 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038671    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe21cc92-8356-4828-834c-854b306ef3c0" containerName="payload-processing"
Jul 30 04:48:21.038695 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038679    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe21cc92-8356-4828-834c-854b306ef3c0" containerName="payload-processing"
Jul 30 04:48:21.038695 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038691    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecc7bb6a-6fa6-4237-84f7-030136d77069" containerName="payload-pre-processing"
Jul 30 04:48:21.038695 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038699    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecc7bb6a-6fa6-4237-84f7-030136d77069" containerName="payload-pre-processing"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038711    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="966e3746-6558-4c91-9329-595ce491b633" containerName="payload-pre-processing"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038718    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="966e3746-6558-4c91-9329-595ce491b633" containerName="payload-pre-processing"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038733    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd46d82e-52b7-4b2e-882d-edbc97582a1d" containerName="payload-processing"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038741    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd46d82e-52b7-4b2e-882d-edbc97582a1d" containerName="payload-processing"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038757    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8bc8f66-bac8-46c9-a3b1-200eb66bbc84" containerName="maas-api"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038765    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8bc8f66-bac8-46c9-a3b1-200eb66bbc84" containerName="maas-api"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038850    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="966e3746-6558-4c91-9329-595ce491b633" containerName="payload-pre-processing"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038862    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd46d82e-52b7-4b2e-882d-edbc97582a1d" containerName="payload-processing"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038873    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe21cc92-8356-4828-834c-854b306ef3c0" containerName="payload-processing"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038882    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c12a586-90ad-49eb-a7f9-d040763f90ad" containerName="payload-processing"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038890    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1275fbc0-ef79-4872-8963-65ba5f4587af" containerName="payload-pre-processing"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038900    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1ebbc43-e61d-4032-a45f-1f4a9a3e90c5" containerName="maas-api"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038908    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cc53b5a-d515-4c21-b7bd-fa28d8c08f84" containerName="maas-api"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038915    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8bc8f66-bac8-46c9-a3b1-200eb66bbc84" containerName="maas-api"
Jul 30 04:48:21.039218 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.038925    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecc7bb6a-6fa6-4237-84f7-030136d77069" containerName="payload-pre-processing"
Jul 30 04:48:21.042252 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.042232    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.044178 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.044157    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qdzjx\"/\"kube-root-ca.crt\""
Jul 30 04:48:21.044704 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.044682    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qdzjx\"/\"openshift-service-ca.crt\""
Jul 30 04:48:21.044819 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.044704    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qdzjx\"/\"default-dockercfg-vdm2s\""
Jul 30 04:48:21.051334 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.051313    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"]
Jul 30 04:48:21.127034 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.127011    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aa93887a-5cd7-4f3b-a28b-614f39f68e46-podres\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.127166 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.127042    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aa93887a-5cd7-4f3b-a28b-614f39f68e46-proc\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.127166 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.127082    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa93887a-5cd7-4f3b-a28b-614f39f68e46-sys\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.127166 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.127104    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa93887a-5cd7-4f3b-a28b-614f39f68e46-lib-modules\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.127166 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.127134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d279b\" (UniqueName: \"kubernetes.io/projected/aa93887a-5cd7-4f3b-a28b-614f39f68e46-kube-api-access-d279b\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.195080 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.195054    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6868887c67-dklsc_ade1b5c1-d49f-40f2-8ac6-54419a36d58b/console/0.log"
Jul 30 04:48:21.228143 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.228108    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aa93887a-5cd7-4f3b-a28b-614f39f68e46-podres\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.228261 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.228152    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aa93887a-5cd7-4f3b-a28b-614f39f68e46-proc\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.228261 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.228213    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa93887a-5cd7-4f3b-a28b-614f39f68e46-sys\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.228261 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.228252    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa93887a-5cd7-4f3b-a28b-614f39f68e46-lib-modules\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.228415 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.228279    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aa93887a-5cd7-4f3b-a28b-614f39f68e46-proc\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.228415 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.228282    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa93887a-5cd7-4f3b-a28b-614f39f68e46-sys\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.228415 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.228303    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d279b\" (UniqueName: \"kubernetes.io/projected/aa93887a-5cd7-4f3b-a28b-614f39f68e46-kube-api-access-d279b\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.228415 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.228254    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aa93887a-5cd7-4f3b-a28b-614f39f68e46-podres\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.228415 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.228337    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa93887a-5cd7-4f3b-a28b-614f39f68e46-lib-modules\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.240916 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.240895    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d279b\" (UniqueName: \"kubernetes.io/projected/aa93887a-5cd7-4f3b-a28b-614f39f68e46-kube-api-access-d279b\") pod \"perf-node-gather-daemonset-mgltw\" (UID: \"aa93887a-5cd7-4f3b-a28b-614f39f68e46\") " pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.257222 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.257205    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-zsxjn_a2ffbdcb-2be1-4bf6-8ab7-1539111e21cc/download-server/0.log"
Jul 30 04:48:21.352207 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.352146    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:21.502368 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:21.502340    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"]
Jul 30 04:48:21.504889 ip-10-0-136-104 kubenswrapper[2569]: W0730 04:48:21.504853    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaa93887a_5cd7_4f3b_a28b_614f39f68e46.slice/crio-0523f26409a24c0d2d27a655062dab572e13359e60f59126b539825d5f9a05ff WatchSource:0}: Error finding container 0523f26409a24c0d2d27a655062dab572e13359e60f59126b539825d5f9a05ff: Status 404 returned error can't find the container with id 0523f26409a24c0d2d27a655062dab572e13359e60f59126b539825d5f9a05ff
Jul 30 04:48:22.444018 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:22.443979    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw" event={"ID":"aa93887a-5cd7-4f3b-a28b-614f39f68e46","Type":"ContainerStarted","Data":"b556ac5b8e54670365d4b074114ebe108ecca6a3a218bf48ec51e97a268ffee4"}
Jul 30 04:48:22.444446 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:22.444028    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw" event={"ID":"aa93887a-5cd7-4f3b-a28b-614f39f68e46","Type":"ContainerStarted","Data":"0523f26409a24c0d2d27a655062dab572e13359e60f59126b539825d5f9a05ff"}
Jul 30 04:48:22.444446 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:22.444104    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:22.460245 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:22.460193    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw" podStartSLOduration=1.46017493 podStartE2EDuration="1.46017493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 04:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:48:22.459166086 +0000 UTC m=+3801.497877108" watchObservedRunningTime="2026-07-30 04:48:22.46017493 +0000 UTC m=+3801.498885951"
Jul 30 04:48:23.154042 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:23.154012    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nth9r_9d01562f-999a-4b59-b5fa-7d6738df7633/dns/0.log"
Jul 30 04:48:23.183885 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:23.183861    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nth9r_9d01562f-999a-4b59-b5fa-7d6738df7633/kube-rbac-proxy/0.log"
Jul 30 04:48:23.332441 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:23.332415    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zdwwk_db777952-f0cd-4473-8c10-5ad3c4d1af11/dns-node-resolver/0.log"
Jul 30 04:48:23.947704 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:23.947677    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5dzsm_4b900747-00b2-49fe-8af7-ab67f091664c/node-ca/0.log"
Jul 30 04:48:25.068922 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:25.068891    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58ccc58984-75hc6_8dd4c125-2623-4e9e-a868-f4ce0157d66a/kube-auth-proxy/0.log"
Jul 30 04:48:25.175915 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:25.175863    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4b7fb_41ac76d3-11aa-4d0d-a44e-d10297c4d5cd/payload-pre-processing/0.log"
Jul 30 04:48:26.043032 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:26.043004    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lzf7c_ad3a8ea4-d784-43ae-832c-16a6ebf99c1f/serve-healthcheck-canary/0.log"
Jul 30 04:48:26.483074 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:26.482999    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-v7mn4_e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2/insights-operator/0.log"
Jul 30 04:48:26.485568 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:26.485545    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-v7mn4_e7f4c995-eb94-4d2d-b1c5-d4b375fec8d2/insights-operator/1.log"
Jul 30 04:48:26.505072 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:26.505052    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-88l4r_ab25f6a4-deba-4c63-a0a7-3be8b62d67c2/kube-rbac-proxy/0.log"
Jul 30 04:48:26.525897 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:26.525877    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-88l4r_ab25f6a4-deba-4c63-a0a7-3be8b62d67c2/exporter/0.log"
Jul 30 04:48:26.547940 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:26.547918    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-88l4r_ab25f6a4-deba-4c63-a0a7-3be8b62d67c2/extractor/0.log"
Jul 30 04:48:28.458851 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:28.458823    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qdzjx/perf-node-gather-daemonset-mgltw"
Jul 30 04:48:29.371477 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:29.371439    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-599789d45f-g7rh7_2eed90ee-243a-45b0-8a41-b13a62f27d0f/manager/0.log"
Jul 30 04:48:29.389946 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:29.389922    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-599789d45f-g7rh7_2eed90ee-243a-45b0-8a41-b13a62f27d0f/copy-manifests/0.log"
Jul 30 04:48:30.752305 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:30.752228    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58ccc58984-75hc6_8dd4c125-2623-4e9e-a868-f4ce0157d66a/kube-auth-proxy/0.log"
Jul 30 04:48:30.838711 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:30.838685    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4b7fb_41ac76d3-11aa-4d0d-a44e-d10297c4d5cd/payload-pre-processing/0.log"
Jul 30 04:48:30.907980 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:30.907956    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-69d4f5b96d-vm9rl_e0e2678d-da59-4b4e-99df-f0059e7f1f56/manager/0.log"
Jul 30 04:48:30.935629 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:30.935609    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-mf6zs_bb619dd6-621f-47c4-a6b1-4d7fb30bbac7/openshift-lws-operator/0.log"
Jul 30 04:48:32.339040 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:32.339012    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58ccc58984-75hc6_8dd4c125-2623-4e9e-a868-f4ce0157d66a/kube-auth-proxy/0.log"
Jul 30 04:48:32.426238 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:32.426212    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4b7fb_41ac76d3-11aa-4d0d-a44e-d10297c4d5cd/payload-pre-processing/0.log"
Jul 30 04:48:35.762560 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:35.762515    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-m8xwx_a7030d89-984c-4160-a612-fb7431545906/migrator/0.log"
Jul 30 04:48:35.783038 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:35.783005    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-m8xwx_a7030d89-984c-4160-a612-fb7431545906/graceful-termination/0.log"
Jul 30 04:48:37.417649 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:37.417621    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mbpx8_621e8000-b5ef-4835-ad07-5fa5cd6e34e5/kube-multus-additional-cni-plugins/0.log"
Jul 30 04:48:37.441253 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:37.441222    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mbpx8_621e8000-b5ef-4835-ad07-5fa5cd6e34e5/egress-router-binary-copy/0.log"
Jul 30 04:48:37.462252 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:37.462235    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mbpx8_621e8000-b5ef-4835-ad07-5fa5cd6e34e5/cni-plugins/0.log"
Jul 30 04:48:37.483329 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:37.483304    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mbpx8_621e8000-b5ef-4835-ad07-5fa5cd6e34e5/bond-cni-plugin/0.log"
Jul 30 04:48:37.506558 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:37.506523    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mbpx8_621e8000-b5ef-4835-ad07-5fa5cd6e34e5/routeoverride-cni/0.log"
Jul 30 04:48:37.527515 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:37.527498    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mbpx8_621e8000-b5ef-4835-ad07-5fa5cd6e34e5/whereabouts-cni-bincopy/0.log"
Jul 30 04:48:37.550674 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:37.550653    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mbpx8_621e8000-b5ef-4835-ad07-5fa5cd6e34e5/whereabouts-cni/0.log"
Jul 30 04:48:37.586674 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:37.586651    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-d6x52_1b178858-acff-4796-87a6-9852811da3c6/kube-multus/0.log"
Jul 30 04:48:37.690407 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:37.690340    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nsf2h_30c27db8-8cfe-492d-855d-4e085771ed3e/network-metrics-daemon/0.log"
Jul 30 04:48:37.708222 ip-10-0-136-104 kubenswrapper[2569]: I0730 04:48:37.708204    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nsf2h_30c27db8-8cfe-492d-855d-4e085771ed3e/kube-rbac-proxy/0.log"